Oct 30 21:30:48.527478 ip-10-0-135-39 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 21:30:48.527492 ip-10-0-135-39 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 21:30:48.527502 ip-10-0-135-39 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 21:30:48.527827 ip-10-0-135-39 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 21:30:58.776395 ip-10-0-135-39 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 21:30:58.776413 ip-10-0-135-39 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0974c4e701d14683ad1ffff81392643f --
Oct 30 21:32:58.703611 ip-10-0-135-39 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:32:59.080846 ip-10-0-135-39 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:32:59.080846 ip-10-0-135-39 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:32:59.080846 ip-10-0-135-39 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:32:59.080846 ip-10-0-135-39 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:32:59.087633 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087562    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.087633 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087632    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087638    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087643    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087647    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087650    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087653    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087656    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087658    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087661    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087664    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087666    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087669    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087671    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087674    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087677    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087679    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087682    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087684    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087687    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.087697 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087689    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087692    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087694    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087697    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087707    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087712    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087714    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087717    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087720    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087722    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087725    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087727    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087730    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087732    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087735    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087738    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087740    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087743    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087745    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.088140 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087748    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087750    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087753    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087755    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087758    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087761    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087763    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087765    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087768    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087770    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087773    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087775    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087778    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087780    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087783    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087786    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087789    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087791    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087794    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087797    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.088592 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087799    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087802    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087804    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087806    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087810    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087812    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087815    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087817    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087820    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087822    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087825    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087827    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087830    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087832    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087834    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087837    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087839    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087842    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087844    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087847    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.089063 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087849    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087852    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087854    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087857    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087859    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087862    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087865    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087868    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087870    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087873    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087875    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087879    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087881    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087884    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087887    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087889    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087892    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087894    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087898    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087901    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.089550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087903    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087906    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087908    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087911    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087913    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087916    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.087918    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089064    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089073    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089077    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089081    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089084    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089086    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089089    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089092    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089095    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089098    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089100    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089103    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.090088 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089106    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089109    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089111    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089114    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089117    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089129    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089132    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089135    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089138    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089141    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089143    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089146    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089149    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089151    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089154    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089156    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089159    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089161    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089164    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089166    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.090611 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089169    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089172    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089175    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089178    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089180    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089183    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089186    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089188    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089191    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089194    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089196    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089199    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089202    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089205    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089208    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089210    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089213    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089215    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089219    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.091082 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089221    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089224    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089227    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089229    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089232    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089234    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089238    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089242    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089246    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089249    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089252    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089255    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089258    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089261    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089263    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089266    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089275    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089278    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089280    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.091533 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089283    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089285    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089288    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089291    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089293    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089296    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089299    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089301    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089304    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089307    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089309    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089312    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089314    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089317    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089320    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089322    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089325    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089327    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089330    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089333    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.091991 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089335    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089338    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089340    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089342    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089345    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089360    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089362    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089365    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089368    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089371    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089374    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089377    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089379    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089382    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089385    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089389    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089880    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089886    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089890    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089894    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.092474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089897    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089899    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089902    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089905    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089908    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089911    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089914    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089917    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089919    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089922    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089924    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089927    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089930    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089932    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089935    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089937    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089940    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089943    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089945    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.092944 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089948    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089950    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089953    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089956    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089959    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089961    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089964    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089966    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089969    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089971    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089974    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089977    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089979    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089982    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089984    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089987    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089990    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089992    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089994    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.089998    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090000    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.093423 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090003    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090005    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090008    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090011    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090013    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090015    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090018    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090020    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090023    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090025    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090027    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090030    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090033    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090035    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090038    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090040    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090042    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090045    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090048    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090051    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.093908 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090053    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090055    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090060    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090063    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090066    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090068    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090071    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090074    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090077    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090079    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090082    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090085    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090087    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090090    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090094    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090096    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090099    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090101    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090104    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.094402 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090106    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090110    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090115    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090118    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090123    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090126    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090129    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090132    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090134    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090136    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090139    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090142    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090144    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090147    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090149    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090153    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090156    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090158    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090161    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.094840 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090165    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090167    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090170    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.090172    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091103    2586 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091111    2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091119    2586 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091124    2586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091128    2586 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091132    2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091136    2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091141    2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091144    2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091147    2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091151    2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091154    2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091157    2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091164    2586 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091167    2586 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091170    2586 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091173    2586 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091176    2586 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091179    2586 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091183    2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:32:59.095327 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091186    2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091189    2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091192    2586 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091196    2586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091200    2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091203    2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091206    2586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091209    2586 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091212    2586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091215    2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091218    2586 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091221    2586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091226    2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091229    2586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091232    2586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091238    2586 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091241    2586 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091244    2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091249    2586 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091252    2586 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091255    2586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091258    2586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091261    2586 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091265    2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:32:59.095902 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091268    2586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091273    2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091276    2586 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091279    2586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091282    2586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091285    2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091288    2586 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091291    2586 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091293    2586 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091296    2586 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091300    2586 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091303    2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091307    2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091311    2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091314    2586 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091317    2586 flags.go:64] FLAG: --help="false"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091320    2586 flags.go:64] FLAG: --hostname-override="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091323    2586 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091326    2586 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091329    2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091332    2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091336    2586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091339    2586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:32:59.096496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091342    2586 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091346    2586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091360    2586 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091364    2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091367    2586 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091370    2586 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091373    2586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091376    2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091379    2586 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091382    2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091386    2586 flags.go:64] FLAG: --lock-file=""
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091389    2586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091392    2586 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091395    2586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091401    2586 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091404    2586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091407    2586 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091410    2586 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091413    2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091416    2586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091419    2586 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091422    2586 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091426    2586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091429    2586 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091433    2586 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:32:59.097039 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091436    2586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091440    2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091442    2586 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091445    2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091448    2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091451    2586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091454    2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091461    2586 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091465    2586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091469    2586 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091472    2586 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091475    2586 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091478    2586 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091481    2586 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091484    2586 flags.go:64] FLAG: --port="10250"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091487    2586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091491    2586 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-09d8040e22cd6e3f9"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091495    2586 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091498    2586 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091501    2586 flags.go:64] FLAG: --register-node="true"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091504    2586 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091508    2586 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091511    2586 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091513    2586 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091516    2586 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091523    2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:32:59.097655 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091526    2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091529    2586 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091532    2586 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091535    2586 flags.go:64] FLAG: --runonce="false"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091539    2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091542    2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091545    2586 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091548    2586 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091551    2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091554    2586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091557    2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091560    2586 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091563    2586 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091566    2586 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091569    2586 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091572    2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091575    2586 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091578    2586 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091581    2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091586    2586 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091589    2586 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091591    2586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091595    2586 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091598    2586 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091601    2586 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:32:59.098289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091604    2586 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091607    2586 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091609    2586 flags.go:64] FLAG: --v="2"
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091613    2586 flags.go:64] FLAG: --version="false"
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091618    2586 flags.go:64] FLAG: --vmodule=""
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091622    2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.091625    2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091759    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091762    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091766    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091769    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091771    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091775    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091779    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091782    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091785    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091787    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091789    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091792    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091794    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091797    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.098895 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091800    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091802    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091805    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091808    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091811    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091813    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091816    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091818    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091821    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091824    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091826    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091829    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091832    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091834    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091837    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091839    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091842    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091844    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091847    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091849    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.099417 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091853    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091856    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091859    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091861    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091864    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091867    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091869    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091873    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091877    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091879    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091882    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091885    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091887    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091890    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091892    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091894    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091897    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091900    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091903    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.099894 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091905    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091907    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091910    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091913    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091916    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091918    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091921    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091923    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091926    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091928    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091931    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091933    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091935    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091939    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091942    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091945    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091947    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091950    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091952    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091955    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.100356 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091957    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091960    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091962    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091965    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091967    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091970    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091973    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091975    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091978    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091980    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091983    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091986    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091988    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091991    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091993    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091996    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.091998    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092001    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092004    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092006    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.100846 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092009    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092012    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092014    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092017    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092019    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092023    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092025    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092028    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092030    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092033    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092035    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092038    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092040    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.092067    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092103    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092108    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.101458 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092111    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092114    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092118    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092122    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092126    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092129    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092132    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092135    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092138    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092140    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092143    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092145    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092148    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092150    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092153    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092155    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092158    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092161    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092163    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.102060 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092166    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092168    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092172    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092175    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092177    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092180    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092182    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092185    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092187    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092190    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092192    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092195    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092198    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092200    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092203    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092206    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092209    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092212    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092214    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092217    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.102604 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092219    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092222    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092225    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092227    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092230    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092232    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092235    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092237    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092240    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092242    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092244    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092247    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092250    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092252    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092256    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092259    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092261    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092264    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092267    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092270    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.103091 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092272    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092275    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092277    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092280    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092282    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092285    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092287    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092291    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092294    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092296    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092299    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092301    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092304    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092306    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092311    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092314    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092317    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092320    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092323    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.103583 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092326    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092329    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092331    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092334    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092337    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092339    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092342    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092345    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092360    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092363    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092366    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092368    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092372    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092374    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092377    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092380    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092382    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092385    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092388    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092390    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.104028 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092394    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092397    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092400    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092402    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092405    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.092407    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.099091    2586 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.099106    2586 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.099711    2586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.099718    2586 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099799    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099804    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099808    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099811    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099814    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099817    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099819    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099822    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099825    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.104550 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099829    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099833    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099843    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099847    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099850    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099854    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099858    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099861    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099863    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099866    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099869    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099872    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099875    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099877    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099880    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099883    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099886    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099888    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099891    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.105003 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099893    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099897    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099900    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099902    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099905    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099908    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099911    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099914    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099916    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099919    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099921    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099924    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099926    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099929    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099931    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099934    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099942    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099945    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099947    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.105539 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099950    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099952    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099955    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099957    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099960    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099963    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099965    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099968    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099970    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099973    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099975    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099978    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099980    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099983    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099987    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099989    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099992    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099994    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099997    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.099999    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.105982 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100002    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100004    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100007    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100009    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100012    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100014    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100017    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100019    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100021    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100024    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100032    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100035    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100037    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100040    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100043    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100045    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100048    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100050    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100052    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100055    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.106474 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100057    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100060    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100062    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100065    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100067    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100070    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100072    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100075    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100078    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100080    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100083    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100085    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100088    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100090    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100092    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100095    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100097    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100100    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.106940 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100102    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.100130    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100339    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100344    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100360    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100371    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100374    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100377    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100379    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100382    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100384    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100387    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100390    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100393    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100395    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100398    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.107366 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100400    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100403    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100421    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100425    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100429    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100432    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100435    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100438    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100440    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100443    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100445    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100448    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100450    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100453    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100455    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100458    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100460    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100463    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100465    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100468    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.107768 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100470    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100472    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100481    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100483    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100486    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100488    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100491    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100493    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100495    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100498    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100500    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100503    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100505    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100508    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100510    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100513    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100516    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100519    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100521    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100524    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.108230 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100526    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100529    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100532    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100534    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100537    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100539    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100542    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100544    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100546    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100549    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100551    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100553    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100556    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100558    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100561    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100569    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100572    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100574    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100576    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100579    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.108711 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100581    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100584    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100586    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100589    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100591    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100594    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100596    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100598    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100601    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100604    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100607    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100609    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100612    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100614    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100617    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100620    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100623    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100628    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.109195 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100631    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100634    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100637    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100640    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100642    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100645    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100647    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100650    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100652    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100654    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100664    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100667    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100669    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100672    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.100696    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:32:59.109629 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100739    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100743    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100746    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100749    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100751    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100754    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100763    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100766    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100773    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100776    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100778    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100781    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100784    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100787    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100790    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100792    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100794    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100797    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100800    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:32:59.109972 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100802    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100804    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100807    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100809    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100812    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100814    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100817    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100819    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100822    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100830    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100833    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100835    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100838    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100841    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100843    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100847    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100851    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100855    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:32:59.110427 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100859    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100862    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100864    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100868    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100870    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100873    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100875    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100878    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100881    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100883    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100886    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100888    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100891    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100893    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100896    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100899    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100901    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100904    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100906    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:32:59.110875 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100909    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100911    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100914    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100916    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100919    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100927    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100929    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100932    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100935    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100938    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100940    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100943    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100945    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100948    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100950    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100953    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100956    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100958    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100961    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100963    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:32:59.111318 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100966    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100968    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100970    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100973    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100975    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100978    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100980    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100983    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100985    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100987    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100990    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100992    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100995    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.100997    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101000    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101002    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101004    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101007    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101015    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101018    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:32:59.111806 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101021    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101023    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101026    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101028    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101031    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101033    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101036    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101040    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101043    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:32:59.101045    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.101846    2586 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.104046    2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.105020    2586 server.go:1014] "Starting client certificate rotation"
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.105109    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.105157    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.106996    2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:32:59.112391 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.108117    2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:32:59.117958 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.117942    2586 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:32:59.121727 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.121712    2586 log.go:25] "Validated CRI v1 image API"
Oct 30 21:32:59.123126 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.123101    2586 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:32:59.125984 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.125965    2586 fs.go:135] Filesystem UUIDs: map[185c5ad3-dcd2-4c95-be36-c4f6f9acdd62:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cd87be85-80f8-4b2e-a581-4273b804eb31:/dev/nvme0n1p4]
Oct 30 21:32:59.126046 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.125983    2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:32:59.131863 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.131754    2586 manager.go:217] Machine: {Timestamp:2026-10-30 21:32:59.130576919 +0000 UTC m=+0.311697271 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3122968 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d29ec51aaa8d5fa4b2a46c9aa4052 SystemUUID:ec2d29ec-51aa-a8d5-fa4b-2a46c9aa4052 BootID:0974c4e7-01d1-4683-ad1f-fff81392643f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:44:83:f3:00:05 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:44:83:f3:00:05 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:7f:51:a2:e2:5c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:32:59.131863 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.131861    2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:32:59.131973 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.131962    2586 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:32:59.133062 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.133039    2586 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:32:59.133203 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.133064    2586 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-39.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:32:59.133244 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.133205    2586 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:32:59.133244 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.133213    2586 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:32:59.133296 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.133246    2586 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:32:59.133296 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.133264    2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:32:59.133969 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.133958    2586 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:32:59.137496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.137484    2586 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:32:59.137604 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.137595    2586 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:32:59.138305 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.138296    2586 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:32:59.138337 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.138310    2586 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:32:59.138337 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.138320    2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:32:59.138337 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.138329    2586 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:32:59.138337 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.138337    2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:32:59.139042 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.139029    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:32:59.141340 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.141305    2586 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:32:59.145399 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.145385    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:32:59.146169 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146158    2586 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:32:59.146214 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146187    2586 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:32:59.146341 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146332    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:32:59.146392 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146364    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:32:59.146392 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146372    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:32:59.146392 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146377    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:32:59.146392 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146383    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:32:59.146392 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146390    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:32:59.146889 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146881    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:32:59.146922 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146890    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:32:59.146922 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146897    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:32:59.146922 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146904    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:32:59.146922 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146912    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:32:59.146922 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.146921    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:32:59.147516 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.147507    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:32:59.147551 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.147517    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:32:59.151141 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.151129    2586 server.go:1263] "Started kubelet"
Oct 30 21:32:59.151215 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.151178    2586 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:32:59.151296 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.151225    2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:32:59.151331 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.151321    2586 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:32:59.151809 ip-10-0-135-39 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:32:59.154577 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.154561    2586 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:32:59.154658 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.154600    2586 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:32:59.157372 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.157329    2586 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:32:59.157895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.157879    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:32:59.158605 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.158590    2586 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:32:59.158706 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.158677    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:32:59.159137 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159119    2586 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:32:59.159137 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159137    2586 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:32:59.159274 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159119    2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:32:59.159274 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159246    2586 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:32:59.159274 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159253    2586 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:32:59.159453 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159318    2586 factory.go:55] Registering systemd factory
Oct 30 21:32:59.159453 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159329    2586 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:32:59.159528 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159516    2586 factory.go:153] Registering CRI-O factory
Oct 30 21:32:59.159528 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159528    2586 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:32:59.159579 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159565    2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:32:59.159608 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159580    2586 factory.go:103] Registering Raw factory
Oct 30 21:32:59.159608 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.159590    2586 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:32:59.160386 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.160342    2586 manager.go:319] Starting recovery of all containers
Oct 30 21:32:59.166182 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.166143    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:32:59.171878 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.171862    2586 manager.go:324] Recovery completed
Oct 30 21:32:59.176980 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.176969    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.178398 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.178377    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.178463 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.178407    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.178463 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.178418    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.178953 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.178934    2586 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:32:59.178953 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.178950    2586 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:32:59.179091 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.178969    2586 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:32:59.179091 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.178981    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.179684 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.179669    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.179759 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.179697    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.179759 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.179709    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.181278 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.181262    2586 policy_none.go:50] "Start"
Oct 30 21:32:59.181278 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.181277    2586 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:32:59.181447 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.181287    2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:32:59.183305 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.183290    2586 policy_none.go:44] "Start"
Oct 30 21:32:59.223480 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.223452    2586 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:32:59.225992 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.225975    2586 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:32:59.226093 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.226004    2586 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:32:59.226093 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.226015    2586 server.go:87] "Starting device plugin registration server"
Oct 30 21:32:59.226257 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.226244    2586 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:32:59.226308 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.226259    2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:32:59.226918 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.226563    2586 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:32:59.226918 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.226670    2586 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:32:59.226918 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.226678    2586 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:32:59.226918 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.226846    2586 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:32:59.226918 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.226887    2586 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:32:59.281966 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.281946    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:32:59.281966 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.281966    2586 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:32:59.282092 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.281985    2586 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:32:59.282092 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.282012    2586 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:32:59.297262 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.297241    2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-39.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:32:59.297262 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.297246    2586 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-39.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:32:59.298120 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.297230    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-39.us-west-1.compute.internal.18e36c2426ea5587  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-39.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-39.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:32:59.151103367 +0000 UTC m=+0.332223713,LastTimestamp:2026-10-30 21:32:59.151103367 +0000 UTC m=+0.332223713,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-39.us-west-1.compute.internal,}"
Oct 30 21:32:59.327115 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.327095    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.328044 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.328028    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.328132 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.328053    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.328132 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.328062    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.328132 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.328082    2586 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.382756 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.382693    2586 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"]
Oct 30 21:32:59.382756 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.382749    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.383532 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.383516    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.383618 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.383542    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.383618 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.383552    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.385700 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.385688    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.385833 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.385818    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.385879 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.385847    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.386330 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.386312    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.386421 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.386346    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.386421 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.386373    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.386495 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.386372    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.386495 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.386469    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.386495 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.386479    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.388449 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.388437    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.388494 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.388459    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.389151 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.389136    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.389226 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.389167    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.389226 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.389182    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.417779 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.417764    2586 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.419735 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.419720    2586 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.422155 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.422140    2586 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:32:59.422201 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.422186    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z5tcw"
Oct 30 21:32:59.452457 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.452424    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-39.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 21:32:59.452457 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.452450    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 21:32:59.452575 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.452476    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 21:32:59.501459 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.501384    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-39.us-west-1.compute.internal.18e36c24288acc5e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-39.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-39.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-39.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:32:59.178396766 +0000 UTC m=+0.359517111,LastTimestamp:2026-10-30 21:32:59.178396766 +0000 UTC m=+0.359517111,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-39.us-west-1.compute.internal,}"
Oct 30 21:32:59.560424 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.560398    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.560513 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.560432    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.560513 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.560453    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a9ea5e58d330910c40e8b399ca003fd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"9a9ea5e58d330910c40e8b399ca003fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.609085 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.609064    2586 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:32:59.609733 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.609714    2586 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-39.us-west-1.compute.internal\" not found" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.660650 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.660604    2586 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-39.us-west-1.compute.internal\" not found" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.661038 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.661022    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.661074 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.661048    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.661074 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.661065    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a9ea5e58d330910c40e8b399ca003fd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"9a9ea5e58d330910c40e8b399ca003fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.661143 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.661092    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a9ea5e58d330910c40e8b399ca003fd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"9a9ea5e58d330910c40e8b399ca003fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.661143 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.661108    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.661143 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.661117    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.719934 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.719912    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.723170 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.723144    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.809987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.809832    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:32:59.810853 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.810836    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:32:59.810942 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.810865    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:32:59.810942 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.810875    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:32:59.810942 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.810906    2586 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.865065 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:32:59.865043    2586 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:32:59.865168 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.865067    2586 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-39.us-west-1.compute.internal\": node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:32:59.975639 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:32:59.975586    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:33:00.076409 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:00.076384    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:33:00.104530 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.104515    2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:33:00.104977 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.104611    2586 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:33:00.104977 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:00.104714    2586 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a87acbcee0c914c0587855f827b89858-cda6e39977984c9a.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-135-39.us-west-1.compute.internal.18e36c24288b2f98\": read tcp 10.0.135.39:58028->18.190.185.126:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-135-39.us-west-1.compute.internal.18e36c24288b2f98  default   5121 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-39.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-39.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-39.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:32:59 +0000 UTC,LastTimestamp:2026-10-30 21:32:59.328066346 +0000 UTC m=+0.509186692,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-39.us-west-1.compute.internal,}"
Oct 30 21:33:00.177001 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:00.176951    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:33:00.277499 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:00.277429    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:33:00.377855 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:00.377823    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:33:00.408945 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.408920    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:00.409951 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.409933    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:00.410038 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.409965    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:00.410038 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.409975    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:00.427439 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.427405    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:28:00 +0000 UTC" deadline="2028-04-18 06:29:55.879274696 +0000 UTC"
Oct 30 21:33:00.427439 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.427436    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12848h56m55.451843787s"
Oct 30 21:33:00.478007 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:00.477982    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:33:00.578171 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:00.578118    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:33:00.579738 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.579714    2586 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:00.659673 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.659649    2586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:33:00.723307 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.723288    2586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:33:00.725217 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.725202    2586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:33:00.781565 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.781536    2586 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:00.830259 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.830186    2586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:33:00.866648 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:00.866610    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf790951ab88ecd279296ee4eb7ed434.slice/crio-81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598 WatchSource:0}: Error finding container 81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598: Status 404 returned error can't find the container with id 81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598
Oct 30 21:33:00.866930 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:00.866906    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9ea5e58d330910c40e8b399ca003fd.slice/crio-fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e WatchSource:0}: Error finding container fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e: Status 404 returned error can't find the container with id fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e
Oct 30 21:33:00.873724 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.873702    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:33:00.924369 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:00.924320    2586 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:01.046320 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.046286    2586 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:01.139768 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.139680    2586 apiserver.go:52] "Watching apiserver"
Oct 30 21:33:01.159323 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.159291    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:33:01.249285 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.249246    2586 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:33:01.249629 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.249605    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-6hsgh","kube-system/global-pull-secret-syncer-2glxt","kube-system/konnectivity-agent-qkddr","openshift-image-registry/node-ca-sswwd","openshift-network-diagnostics/network-check-target-b6gtf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b","openshift-cluster-node-tuning-operator/tuned-24q2x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-s5wqs","openshift-multus/multus-svcjf","openshift-multus/network-metrics-daemon-t6n8r","kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal","openshift-dns/node-resolver-zmw2g","openshift-network-operator/iptables-alerter-dgbt5"]
Oct 30 21:33:01.253692 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.253671    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.255897 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.255872    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.258168 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.258139    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.260408 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.260380    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.262685 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.262665    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:01.262791 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.262737    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:01.265014 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.264997    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.267482 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.267459    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.268113 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268095    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49415b6f-832d-4e10-b7f9-e0b773574b30-agent-certs\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.268198 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268119    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19c8fb74-659e-4087-a035-3f26fa17b33c-serviceca\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.268198 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268135    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-modprobe-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268198 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268150    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-conf\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268198 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268170    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-var-lib-kubelet\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268333 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268249    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-tmp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268333 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268302    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrkq\" (UniqueName: \"kubernetes.io/projected/19c8fb74-659e-4087-a035-3f26fa17b33c-kube-api-access-clrkq\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.268333 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268329    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-host\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268448 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268384    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysconfig\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268448 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268414    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268448 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268438    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-run\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268547 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268469    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-lib-modules\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268547 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268524    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87zqp\" (UniqueName: \"kubernetes.io/projected/b66918c1-6a0a-495b-bb2e-df4e61590fe4-kube-api-access-87zqp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268612 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268557    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49415b6f-832d-4e10-b7f9-e0b773574b30-konnectivity-ca\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.268645 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268616    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-kubernetes\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268684 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268645    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6157882-1c00-4bb2-9790-25ba55f8ceb7-tmp-dir\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.268717 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268700    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9nz\" (UniqueName: \"kubernetes.io/projected/e6157882-1c00-4bb2-9790-25ba55f8ceb7-kube-api-access-lx9nz\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.268748 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268734    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c8fb74-659e-4087-a035-3f26fa17b33c-host\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.268781 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268771    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-systemd\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268815 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268798    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-sys\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268857 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268840    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-tuned\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.268892 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.268872    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6157882-1c00-4bb2-9790-25ba55f8ceb7-hosts-file\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.271525 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.271500    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.273037 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.273016    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.273141 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.273126    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:01.276094 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.276072    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.278460 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.278430    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:01.278535 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.278509    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:01.280816 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.280798    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.286343 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.286292    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"9a9ea5e58d330910c40e8b399ca003fd","Type":"ContainerStarted","Data":"fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e"}
Oct 30 21:33:01.287404 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.287376    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"af790951ab88ecd279296ee4eb7ed434","Type":"ContainerStarted","Data":"81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598"}
Oct 30 21:33:01.309643 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.309619    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:33:01.309745 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.309619    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:33:01.309745 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.309702    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6t698\""
Oct 30 21:33:01.311048 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.311020    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:33:01.311117 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.311049    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:33:01.311158 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.311127    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mzgn4\""
Oct 30 21:33:01.313130 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.313113    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:33:01.313366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.313347    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:33:01.313486 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.313471    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wn5fv\""
Oct 30 21:33:01.315898 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.315881    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:33:01.315993 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.315979    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:33:01.316074 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.316055    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:33:01.316115 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.316109    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2v7jg\""
Oct 30 21:33:01.320527 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.320513    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:33:01.320591 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.320561    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:33:01.320633 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.320618    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:33:01.320666 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.320649    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pplxq\""
Oct 30 21:33:01.322725 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.322710    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rjmnn\""
Oct 30 21:33:01.323522 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.323508    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:33:01.323629 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.323616    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:33:01.323795 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.323775    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:33:01.323886 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.323873    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:33:01.358496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.358464    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:33:01.358496 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.358466    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:33:01.358715 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.358558    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:33:01.358766 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.358723    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:33:01.360525 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.360510    2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:33:01.362772 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.362756    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:33:01.362863 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.362831    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:33:01.362863 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.362842    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:33:01.362976 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.362929    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:33:01.363074 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363052    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jnpcj\""
Oct 30 21:33:01.363199 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363178    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:33:01.363260 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363241    2586 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:33:01.363439 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363420    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kwnrb"
Oct 30 21:33:01.363641 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363626    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tvlkp\""
Oct 30 21:33:01.363689 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363673    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:33:01.363748 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363725    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zs2mb\""
Oct 30 21:33:01.363957 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.363945    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:33:01.369389 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369348    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-kubernetes\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.369481 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369417    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.369481 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369452    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mb9t\" (UniqueName: \"kubernetes.io/projected/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-kube-api-access-9mb9t\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.369561 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369482    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-var-lib-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.369561 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369509    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l648n\" (UniqueName: \"kubernetes.io/projected/23c665c7-85e7-4249-87e6-412b16d9e317-kube-api-access-l648n\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.369561 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369541    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-kubernetes\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.369561 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369551    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-sys\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.369700 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369589    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-kubelet\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.369700 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369608    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-slash\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.369700 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369601    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-sys\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.369700 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369643    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-conf\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.369700 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369673    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-system-cni-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.369700 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369688    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cnibin\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369706    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-registration-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369729    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0ceab3b-942e-4950-bb5e-96f62e431365-ovn-node-metrics-cert\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369744    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-system-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369759    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-cni-binary-copy\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369785    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-bin\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369799    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-conf\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369837    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49415b6f-832d-4e10-b7f9-e0b773574b30-konnectivity-ca\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369859    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-kubelet-config\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.369905 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369882    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c8fb74-659e-4087-a035-3f26fa17b33c-host\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.370177 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369954    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c8fb74-659e-4087-a035-3f26fa17b33c-host\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.370177 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.369979    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-binary-copy\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.370177 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370074    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-config\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.370177 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370099    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-socket-dir-parent\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.370177 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370116    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-multus\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.370177 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370140    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-etc-kubernetes\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.370366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370175    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-tuned\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.370366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370206    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6157882-1c00-4bb2-9790-25ba55f8ceb7-hosts-file\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.370366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370252    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-script-lib\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.370366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370260    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6157882-1c00-4bb2-9790-25ba55f8ceb7-hosts-file\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.370366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370293    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-modprobe-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.370366 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370328    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bczv\" (UniqueName: \"kubernetes.io/projected/e0ceab3b-942e-4950-bb5e-96f62e431365-kube-api-access-8bczv\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370370    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-829q5\" (UniqueName: \"kubernetes.io/projected/8953ad86-e744-4251-99cc-278bc81abe74-kube-api-access-829q5\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370378    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-modprobe-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370387    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-host\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370409    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysconfig\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370428    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370441    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-host\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370444    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-device-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370484    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-netns\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370487    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysconfig\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370522    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-netd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370531    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49415b6f-832d-4e10-b7f9-e0b773574b30-konnectivity-ca\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.370571 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370543    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-netns\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.370931 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370578    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6157882-1c00-4bb2-9790-25ba55f8ceb7-tmp-dir\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.370931 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370604    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:01.370931 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370668    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-multus-daemon-config\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.370931 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370689    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19c8fb74-659e-4087-a035-3f26fa17b33c-serviceca\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.370931 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370706    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.370931 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370780    2586 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:33:01.370931 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.370826    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6157882-1c00-4bb2-9790-25ba55f8ceb7-tmp-dir\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.371218 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371197    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.371255 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371221    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19c8fb74-659e-4087-a035-3f26fa17b33c-serviceca\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.371255 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371235    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.371316 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371291    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-cnibin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.371316 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371308    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-conf-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.371374 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371330    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-iptables-alerter-script\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.371374 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371353    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-host-slash\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.371374 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371370    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd5bx\" (UniqueName: \"kubernetes.io/projected/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kube-api-access-kd5bx\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.371491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371385    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.371491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371419    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-var-lib-kubelet\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371440    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-tmp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371456    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-dbus\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.371491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371471    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-socket-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.371491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371473    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-var-lib-kubelet\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371735 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371493    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:01.371735 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371536    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-systemd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.371735 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371574    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-etc-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.371735 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371598    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371735 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371672    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371735 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371701    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-run\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371738    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-lib-modules\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371755    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87zqp\" (UniqueName: \"kubernetes.io/projected/b66918c1-6a0a-495b-bb2e-df4e61590fe4-kube-api-access-87zqp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371785    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371811    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-run\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371808    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-sys-fs\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371834    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-lib-modules\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371927    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9nz\" (UniqueName: \"kubernetes.io/projected/e6157882-1c00-4bb2-9790-25ba55f8ceb7-kube-api-access-lx9nz\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371944    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-k8s-cni-cncf-io\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371960    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-multus-certs\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.371987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.371976    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-systemd\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372074    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49415b6f-832d-4e10-b7f9-e0b773574b30-agent-certs\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372157    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-ovn\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372176    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-systemd\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372184    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-bin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372208    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-hostroot\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372232    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrkq\" (UniqueName: \"kubernetes.io/projected/19c8fb74-659e-4087-a035-3f26fa17b33c-kube-api-access-clrkq\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372259    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372284    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-systemd-units\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372328    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-log-socket\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.372393 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372354    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-env-overrides\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.372761 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372396    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctzfj\" (UniqueName: \"kubernetes.io/projected/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-kube-api-access-ctzfj\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.372761 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372434    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.372761 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372464    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.372761 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372486    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-node-log\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.372761 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.372510    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-kubelet\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.374582 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.374563    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-tmp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.374659 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.374582    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-tuned\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.374789 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.374773    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49415b6f-832d-4e10-b7f9-e0b773574b30-agent-certs\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.433753 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.433673    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrkq\" (UniqueName: \"kubernetes.io/projected/19c8fb74-659e-4087-a035-3f26fa17b33c-kube-api-access-clrkq\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.433887 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.433773    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9nz\" (UniqueName: \"kubernetes.io/projected/e6157882-1c00-4bb2-9790-25ba55f8ceb7-kube-api-access-lx9nz\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.433887 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.433816    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87zqp\" (UniqueName: \"kubernetes.io/projected/b66918c1-6a0a-495b-bb2e-df4e61590fe4-kube-api-access-87zqp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.473697 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473662    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-829q5\" (UniqueName: \"kubernetes.io/projected/8953ad86-e744-4251-99cc-278bc81abe74-kube-api-access-829q5\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:01.473802 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473704    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.473802 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473726    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-device-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.473802 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473749    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-netns\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.473802 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473772    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-netd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.473802 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473798    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-netns\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473826    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473827    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473850    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-multus-daemon-config\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473855    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-netd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473829    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-netns\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473871    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-netns\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473875    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473828    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-device-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473924    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473951    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473977    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-cnibin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474000    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-conf-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474012 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.473996    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474045    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-conf-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474059    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474072    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-iptables-alerter-script\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474056    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-cnibin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474097    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-host-slash\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474120    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd5bx\" (UniqueName: \"kubernetes.io/projected/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kube-api-access-kd5bx\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474144    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474164    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474188    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474207    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-host-slash\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474248    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-dbus\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474278    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-socket-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474308    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474312    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-dbus\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474334    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-systemd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474359    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-etc-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.474703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474423    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-systemd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474420    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474450    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-socket-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474457    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-etc-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474470    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-sys-fs\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474490    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-k8s-cni-cncf-io\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474507    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-multus-certs\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.474520    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474528    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-ovn\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474540    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-sys-fs\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474554    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-bin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474565    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-multus-certs\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474569    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-k8s-cni-cncf-io\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474573    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-ovn\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.474590    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs podName:8953ad86-e744-4251-99cc-278bc81abe74 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:01.97456787 +0000 UTC m=+3.155688233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs") pod "network-metrics-daemon-t6n8r" (UID: "8953ad86-e744-4251-99cc-278bc81abe74") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474603    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-bin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474612    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-hostroot\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.475309 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474633    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-hostroot\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474646    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474681    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-systemd-units\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474724    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-systemd-units\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.474750    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474791    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-log-socket\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.474801    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret podName:1a7bdebe-605a-432d-8cc7-91fc5417158d nodeName:}" failed. No retries permitted until 2026-10-30 21:33:01.974788806 +0000 UTC m=+3.155909155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret") pod "global-pull-secret-syncer-2glxt" (UID: "1a7bdebe-605a-432d-8cc7-91fc5417158d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474815    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-log-socket\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474821    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-env-overrides\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474839    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctzfj\" (UniqueName: \"kubernetes.io/projected/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-kube-api-access-ctzfj\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474861    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474878    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474895    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-node-log\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474918    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-kubelet\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474960    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474968    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-kubelet\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474979    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-multus-daemon-config\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.476068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474993    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.474996    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475012    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-node-log\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475031    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mb9t\" (UniqueName: \"kubernetes.io/projected/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-kube-api-access-9mb9t\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475057    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-var-lib-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475074    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l648n\" (UniqueName: \"kubernetes.io/projected/23c665c7-85e7-4249-87e6-412b16d9e317-kube-api-access-l648n\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475095    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-kubelet\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475125    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-var-lib-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475136    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-kubelet\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475232    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-slash\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475265    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-system-cni-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475268    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-slash\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475291    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cnibin\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475310    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-system-cni-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475317    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-registration-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475345    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0ceab3b-942e-4950-bb5e-96f62e431365-ovn-node-metrics-cert\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475348    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cnibin\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.476812 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475387    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-system-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475402    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-registration-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475413    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-cni-binary-copy\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475443    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-bin\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475469    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-kubelet-config\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475456    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-system-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475509    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-binary-copy\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475534    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-config\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475535    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-bin\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475580    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-kubelet-config\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475680    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-iptables-alerter-script\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475819    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-env-overrides\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475946    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-socket-dir-parent\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.475980    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-multus\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476011    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-etc-kubernetes\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476042    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-script-lib\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476063    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-socket-dir-parent\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477299 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476072    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bczv\" (UniqueName: \"kubernetes.io/projected/e0ceab3b-942e-4950-bb5e-96f62e431365-kube-api-access-8bczv\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476077    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-cni-binary-copy\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476142    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476165    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-binary-copy\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476191    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-config\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476194    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-etc-kubernetes\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476217    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-multus\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476596    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.477803 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.476684    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-script-lib\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.478031 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.477864    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0ceab3b-942e-4950-bb5e-96f62e431365-ovn-node-metrics-cert\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.537616 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.537581    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:01.537616 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.537616    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:01.537836 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.537632    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8q5kq for pod openshift-network-diagnostics/network-check-target-b6gtf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:01.537836 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.537727    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq podName:d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:02.037706158 +0000 UTC m=+3.218826514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8q5kq" (UniqueName: "kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq") pod "network-check-target-b6gtf" (UID: "d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:01.537966 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.537894    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd5bx\" (UniqueName: \"kubernetes.io/projected/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kube-api-access-kd5bx\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.538372 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.538342    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-829q5\" (UniqueName: \"kubernetes.io/projected/8953ad86-e744-4251-99cc-278bc81abe74-kube-api-access-829q5\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:01.538673 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.538650    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l648n\" (UniqueName: \"kubernetes.io/projected/23c665c7-85e7-4249-87e6-412b16d9e317-kube-api-access-l648n\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.538743 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.538722    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctzfj\" (UniqueName: \"kubernetes.io/projected/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-kube-api-access-ctzfj\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.539823 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.539802    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bczv\" (UniqueName: \"kubernetes.io/projected/e0ceab3b-942e-4950-bb5e-96f62e431365-kube-api-access-8bczv\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.539915 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.539900    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mb9t\" (UniqueName: \"kubernetes.io/projected/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-kube-api-access-9mb9t\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.566843 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.566823    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:33:01.572316 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.572303    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:33:01.577160 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.577140    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66918c1_6a0a_495b_bb2e_df4e61590fe4.slice/crio-14e22e66372f99a4b54152cd0d1224541155f8b6b7034c506728246116478462 WatchSource:0}: Error finding container 14e22e66372f99a4b54152cd0d1224541155f8b6b7034c506728246116478462: Status 404 returned error can't find the container with id 14e22e66372f99a4b54152cd0d1224541155f8b6b7034c506728246116478462
Oct 30 21:33:01.581482 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.581463    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:01.581736 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.581717    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6157882_1c00_4bb2_9790_25ba55f8ceb7.slice/crio-2fe3dbecc316b7419d6178e32b7e772cd2779564da2434ee3c876b3b4c7d2f85 WatchSource:0}: Error finding container 2fe3dbecc316b7419d6178e32b7e772cd2779564da2434ee3c876b3b4c7d2f85: Status 404 returned error can't find the container with id 2fe3dbecc316b7419d6178e32b7e772cd2779564da2434ee3c876b3b4c7d2f85
Oct 30 21:33:01.587439 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.587423    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:33:01.590346 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.590329    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49415b6f_832d_4e10_b7f9_e0b773574b30.slice/crio-3ba54b3e04e6c6dbfa54a19250ff49288928c615fd4262ba9b844101e183ad4b WatchSource:0}: Error finding container 3ba54b3e04e6c6dbfa54a19250ff49288928c615fd4262ba9b844101e183ad4b: Status 404 returned error can't find the container with id 3ba54b3e04e6c6dbfa54a19250ff49288928c615fd4262ba9b844101e183ad4b
Oct 30 21:33:01.596618 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.596598    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c8fb74_659e_4087_a035_3f26fa17b33c.slice/crio-8cd6c3896abdef6848cab2b2a64c300a6767576463f7112c45974d9f82e82e2d WatchSource:0}: Error finding container 8cd6c3896abdef6848cab2b2a64c300a6767576463f7112c45974d9f82e82e2d: Status 404 returned error can't find the container with id 8cd6c3896abdef6848cab2b2a64c300a6767576463f7112c45974d9f82e82e2d
Oct 30 21:33:01.597859 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.597835    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:33:01.606454 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.606423    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df4aa44_fd47_49f0_9ded_df9f9ad8fb8f.slice/crio-40ec291de57f9ebe37fc138551f9879af974ab785cde76cf5a9387015d84cc42 WatchSource:0}: Error finding container 40ec291de57f9ebe37fc138551f9879af974ab785cde76cf5a9387015d84cc42: Status 404 returned error can't find the container with id 40ec291de57f9ebe37fc138551f9879af974ab785cde76cf5a9387015d84cc42
Oct 30 21:33:01.617586 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.617567    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-svcjf"
Oct 30 21:33:01.622529 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.622508    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:33:01.624827 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.624806    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:33:01.626372 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.626350    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c665c7_85e7_4249_87e6_412b16d9e317.slice/crio-a000f4d35b63ea8cd6cd3e6b8283f54bba3b2b7e1dba6ccb9f125364034e41c1 WatchSource:0}: Error finding container a000f4d35b63ea8cd6cd3e6b8283f54bba3b2b7e1dba6ccb9f125364034e41c1: Status 404 returned error can't find the container with id a000f4d35b63ea8cd6cd3e6b8283f54bba3b2b7e1dba6ccb9f125364034e41c1
Oct 30 21:33:01.632702 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.632679    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5cb4be6_9a9a_4318_bdb3_883bcbcffc15.slice/crio-98e9824226645370648f6836cc0079e64161483d6eca28a14bac0313481c773a WatchSource:0}: Error finding container 98e9824226645370648f6836cc0079e64161483d6eca28a14bac0313481c773a: Status 404 returned error can't find the container with id 98e9824226645370648f6836cc0079e64161483d6eca28a14bac0313481c773a
Oct 30 21:33:01.634346 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.634326    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43da839_7e5b_4fe8_aa90_06fa9ee3fbc1.slice/crio-4f8b281f87588eff41948af47992acdd755673eed00d037b961ad8ab032dd662 WatchSource:0}: Error finding container 4f8b281f87588eff41948af47992acdd755673eed00d037b961ad8ab032dd662: Status 404 returned error can't find the container with id 4f8b281f87588eff41948af47992acdd755673eed00d037b961ad8ab032dd662
Oct 30 21:33:01.634820 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.634770    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:01.644014 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:01.643989    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ceab3b_942e_4950_bb5e_96f62e431365.slice/crio-8d0c954fbe02bdfbb278e454964a537b1d13b54760b3cf81a7232c36c75bc181 WatchSource:0}: Error finding container 8d0c954fbe02bdfbb278e454964a537b1d13b54760b3cf81a7232c36c75bc181: Status 404 returned error can't find the container with id 8d0c954fbe02bdfbb278e454964a537b1d13b54760b3cf81a7232c36c75bc181
Oct 30 21:33:01.979600 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.979563    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:01.979792 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:01.979626    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:01.979792 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.979781    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:01.979906 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.979848    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret podName:1a7bdebe-605a-432d-8cc7-91fc5417158d nodeName:}" failed. No retries permitted until 2026-10-30 21:33:02.979828741 +0000 UTC m=+4.160949090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret") pod "global-pull-secret-syncer-2glxt" (UID: "1a7bdebe-605a-432d-8cc7-91fc5417158d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:01.979966 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.979922    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:01.979966 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:01.979961    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs podName:8953ad86-e744-4251-99cc-278bc81abe74 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:02.979948566 +0000 UTC m=+4.161068914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs") pod "network-metrics-daemon-t6n8r" (UID: "8953ad86-e744-4251-99cc-278bc81abe74") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:02.080733 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.080699    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:02.080924 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.080901    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:02.080997 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.080934    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:02.080997 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.080947    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8q5kq for pod openshift-network-diagnostics/network-check-target-b6gtf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:02.081084 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.081011    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq podName:d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:03.080991605 +0000 UTC m=+4.262111941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q5kq" (UniqueName: "kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq") pod "network-check-target-b6gtf" (UID: "d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:02.291230 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.291138    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-svcjf" event={"ID":"23c665c7-85e7-4249-87e6-412b16d9e317","Type":"ContainerStarted","Data":"a000f4d35b63ea8cd6cd3e6b8283f54bba3b2b7e1dba6ccb9f125364034e41c1"}
Oct 30 21:33:02.292840 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.292806    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b" event={"ID":"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f","Type":"ContainerStarted","Data":"40ec291de57f9ebe37fc138551f9879af974ab785cde76cf5a9387015d84cc42"}
Oct 30 21:33:02.294276 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.294249    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sswwd" event={"ID":"19c8fb74-659e-4087-a035-3f26fa17b33c","Type":"ContainerStarted","Data":"8cd6c3896abdef6848cab2b2a64c300a6767576463f7112c45974d9f82e82e2d"}
Oct 30 21:33:02.295932 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.295906    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"8d0c954fbe02bdfbb278e454964a537b1d13b54760b3cf81a7232c36c75bc181"}
Oct 30 21:33:02.297331 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.297298    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerStarted","Data":"98e9824226645370648f6836cc0079e64161483d6eca28a14bac0313481c773a"}
Oct 30 21:33:02.298858 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.298835    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qkddr" event={"ID":"49415b6f-832d-4e10-b7f9-e0b773574b30","Type":"ContainerStarted","Data":"3ba54b3e04e6c6dbfa54a19250ff49288928c615fd4262ba9b844101e183ad4b"}
Oct 30 21:33:02.300257 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.300234    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zmw2g" event={"ID":"e6157882-1c00-4bb2-9790-25ba55f8ceb7","Type":"ContainerStarted","Data":"2fe3dbecc316b7419d6178e32b7e772cd2779564da2434ee3c876b3b4c7d2f85"}
Oct 30 21:33:02.301839 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.301817    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24q2x" event={"ID":"b66918c1-6a0a-495b-bb2e-df4e61590fe4","Type":"ContainerStarted","Data":"14e22e66372f99a4b54152cd0d1224541155f8b6b7034c506728246116478462"}
Oct 30 21:33:02.303591 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.303564    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dgbt5" event={"ID":"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1","Type":"ContainerStarted","Data":"4f8b281f87588eff41948af47992acdd755673eed00d037b961ad8ab032dd662"}
Oct 30 21:33:02.366478 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.366386    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:28:02 +0000 UTC" deadline="2028-06-03 16:38:39.869681956 +0000 UTC"
Oct 30 21:33:02.366478 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.366440    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13963h5m37.503245145s"
Oct 30 21:33:02.987979 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.987945    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:02.988164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:02.988047    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:02.988243 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.988173    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:02.988243 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.988233    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs podName:8953ad86-e744-4251-99cc-278bc81abe74 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:04.988214692 +0000 UTC m=+6.169335032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs") pod "network-metrics-daemon-t6n8r" (UID: "8953ad86-e744-4251-99cc-278bc81abe74") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:02.988359 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.988302    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:02.988359 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:02.988337    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret podName:1a7bdebe-605a-432d-8cc7-91fc5417158d nodeName:}" failed. No retries permitted until 2026-10-30 21:33:04.988326145 +0000 UTC m=+6.169446485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret") pod "global-pull-secret-syncer-2glxt" (UID: "1a7bdebe-605a-432d-8cc7-91fc5417158d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:03.088369 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:03.088335    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:03.088562 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:03.088522    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:03.088562 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:03.088543    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:03.088562 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:03.088559    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8q5kq for pod openshift-network-diagnostics/network-check-target-b6gtf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:03.088680 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:03.088617    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq podName:d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:05.088599245 +0000 UTC m=+6.269719581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q5kq" (UniqueName: "kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq") pod "network-check-target-b6gtf" (UID: "d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:03.284936 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:03.284861    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:03.285091 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:03.285002    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:03.285164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:03.285101    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:03.285216 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:03.285176    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:03.285269 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:03.285241    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:03.285335 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:03.285315    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:03.371104 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:03.371036    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:28:02 +0000 UTC" deadline="2028-05-08 07:07:51.679564737 +0000 UTC"
Oct 30 21:33:03.371104 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:03.371073    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13329h34m48.308495609s"
Oct 30 21:33:05.009808 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.008714    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:05.009808 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.009035    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:05.009808 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.009197    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:05.009808 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.009279    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret podName:1a7bdebe-605a-432d-8cc7-91fc5417158d nodeName:}" failed. No retries permitted until 2026-10-30 21:33:09.009259937 +0000 UTC m=+10.190380276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret") pod "global-pull-secret-syncer-2glxt" (UID: "1a7bdebe-605a-432d-8cc7-91fc5417158d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:05.009808 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.009716    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:05.009808 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.009772    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs podName:8953ad86-e744-4251-99cc-278bc81abe74 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:09.009754318 +0000 UTC m=+10.190874657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs") pod "network-metrics-daemon-t6n8r" (UID: "8953ad86-e744-4251-99cc-278bc81abe74") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:05.109486 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.109458    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:05.109638 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.109621    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:05.109732 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.109647    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:05.109732 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.109661    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8q5kq for pod openshift-network-diagnostics/network-check-target-b6gtf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:05.109732 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.109718    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq podName:d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:09.109700093 +0000 UTC m=+10.290820443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q5kq" (UniqueName: "kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq") pod "network-check-target-b6gtf" (UID: "d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:05.285067 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.285039    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:05.285191 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.285158    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:05.285589 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.285567    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:05.285712 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.285679    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:05.285781 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.285744    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:05.285839 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:05.285821    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:05.340195 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.339570    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"9a9ea5e58d330910c40e8b399ca003fd","Type":"ContainerStarted","Data":"0659f086608e12816c7c1d86609ae661b9d971bae1048a391a64d66d71d1909f"}
Oct 30 21:33:05.344969 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.344938    2586 generic.go:356] "Generic (PLEG): container finished" podID="af790951ab88ecd279296ee4eb7ed434" containerID="95d6d71dec84e692eee52bb89519bd5568ed4b92f16da144ad5c47a2900e2b66" exitCode=0
Oct 30 21:33:05.345086 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.345027    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"af790951ab88ecd279296ee4eb7ed434","Type":"ContainerDied","Data":"95d6d71dec84e692eee52bb89519bd5568ed4b92f16da144ad5c47a2900e2b66"}
Oct 30 21:33:05.352396 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.352369    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerStarted","Data":"99d84d1eed2c865ba4a695e4b869c81d1b32252545f1cd9571829eb1783c3a7b"}
Oct 30 21:33:05.465703 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:05.464927    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" podStartSLOduration=4.464904927 podStartE2EDuration="4.464904927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:05.464394217 +0000 UTC m=+6.645514576" watchObservedRunningTime="2026-10-30 21:33:05.464904927 +0000 UTC m=+6.646025284"
Oct 30 21:33:06.365181 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:06.365142    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"af790951ab88ecd279296ee4eb7ed434","Type":"ContainerStarted","Data":"d668f3673fcae9931f244336c382658bcd0863456ea10d84697aa3771075d93f"}
Oct 30 21:33:06.370357 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:06.370322    2586 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="99d84d1eed2c865ba4a695e4b869c81d1b32252545f1cd9571829eb1783c3a7b" exitCode=0
Oct 30 21:33:06.371374 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:06.371349    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerDied","Data":"99d84d1eed2c865ba4a695e4b869c81d1b32252545f1cd9571829eb1783c3a7b"}
Oct 30 21:33:06.488999 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:06.488932    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal" podStartSLOduration=5.488914252 podStartE2EDuration="5.488914252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:06.488358163 +0000 UTC m=+7.669478524" watchObservedRunningTime="2026-10-30 21:33:06.488914252 +0000 UTC m=+7.670034613"
Oct 30 21:33:07.284588 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:07.284553    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:07.284773 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:07.284706    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:07.285015 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:07.284983    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:07.285122 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:07.285067    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:07.285122 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:07.285115    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:07.285230 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:07.285163    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:07.378485 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:07.378448    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qkddr" event={"ID":"49415b6f-832d-4e10-b7f9-e0b773574b30","Type":"ContainerStarted","Data":"71f953ff4294b08d74061fb56745e0a3df2e5984deeaffccb73b6fad2cdc8679"}
Oct 30 21:33:08.384784 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:08.384691    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b" event={"ID":"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f","Type":"ContainerStarted","Data":"a3f3bf8974bb65d6539eaf40014d72609a8e3f7edd13bbee431661e0e96c7730"}
Oct 30 21:33:08.388582 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:08.388502    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sswwd" event={"ID":"19c8fb74-659e-4087-a035-3f26fa17b33c","Type":"ContainerStarted","Data":"c5a2e8e42643b2c9b2e66ef434ab4ec12494e39faba0a920802bb4517a233163"}
Oct 30 21:33:08.513661 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:08.513157    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qkddr" podStartSLOduration=3.58448155 podStartE2EDuration="8.513135105s" totalImagesPullingTime="4.928653555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:07.49927128 +0000 UTC m=+8.680391639" watchObservedRunningTime="2026-10-30 21:33:08.513135105 +0000 UTC m=+9.694255481"
Oct 30 21:33:08.513661 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:08.513276    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sswwd" podStartSLOduration=2.501354209 podStartE2EDuration="8.513264287s" totalImagesPullingTime="6.011910078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:08.512652508 +0000 UTC m=+9.693772868" watchObservedRunningTime="2026-10-30 21:33:08.513264287 +0000 UTC m=+9.694384646"
Oct 30 21:33:08.530247 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:08.530088    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:08.530798 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:08.530625    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:09.043757 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:09.042993    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:09.043757 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:09.043046    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:09.043757 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.043176    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:09.043757 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.043238    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret podName:1a7bdebe-605a-432d-8cc7-91fc5417158d nodeName:}" failed. No retries permitted until 2026-10-30 21:33:17.043219432 +0000 UTC m=+18.224339782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret") pod "global-pull-secret-syncer-2glxt" (UID: "1a7bdebe-605a-432d-8cc7-91fc5417158d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:09.043757 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.043666    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:09.043757 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.043716    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs podName:8953ad86-e744-4251-99cc-278bc81abe74 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:17.043700741 +0000 UTC m=+18.224821091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs") pod "network-metrics-daemon-t6n8r" (UID: "8953ad86-e744-4251-99cc-278bc81abe74") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:09.144267 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:09.143960    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:09.144267 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.144137    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:09.144267 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.144158    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:09.144267 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.144172    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8q5kq for pod openshift-network-diagnostics/network-check-target-b6gtf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:09.144267 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.144228    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq podName:d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:17.144210019 +0000 UTC m=+18.325330368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q5kq" (UniqueName: "kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq") pod "network-check-target-b6gtf" (UID: "d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:09.284610 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:09.284112    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:09.284610 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.284240    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:09.284610 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:09.284519    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:09.284956 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.284838    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:09.286805 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:09.285643    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:09.286805 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:09.285761    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:10.394564 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:10.394540    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:33:10.525713 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:10.525691    2586 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:33:11.236268 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.235990    2586 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:33:10.525711595Z","UUID":"978bf6a1-7b3e-4bd9-82d4-a0eefee66adb","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:33:11.238119 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.238094    2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:33:11.238238 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.238129    2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:33:11.283132 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.283104    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:11.283132 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.283145    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:11.283351 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:11.283246    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:11.283351 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:11.283335    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:11.283473 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.283391    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:11.283529 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:11.283480    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:11.399935 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.399514    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zmw2g" event={"ID":"e6157882-1c00-4bb2-9790-25ba55f8ceb7","Type":"ContainerStarted","Data":"fdaf88c362bde7166c7451f6a5511d2e7df39d6a4b5df3bc1c8d384ff5f7f2c5"}
Oct 30 21:33:11.401425 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.401390    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-svcjf" event={"ID":"23c665c7-85e7-4249-87e6-412b16d9e317","Type":"ContainerStarted","Data":"94f4582ffda13e3b601bfa7c126fa58e949d739c3097b10d47955d08f134fe84"}
Oct 30 21:33:11.403821 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.403794    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b" event={"ID":"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f","Type":"ContainerStarted","Data":"234d0a40392b9d4d9cae5e7d6cf47792d265db6dc8047d05c7d89c94861bf6bf"}
Oct 30 21:33:11.406329 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.406305    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerStarted","Data":"1390a7d37fe5cc41f320afaa8ecacb0a3017d6f24f9c96572b9f2e43e13c6d9f"}
Oct 30 21:33:11.525159 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:11.524629    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-zmw2g" podStartSLOduration=2.7440217970000003 podStartE2EDuration="11.524606307s" totalImagesPullingTime="8.78058451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:11.522627981 +0000 UTC m=+12.703748351" watchObservedRunningTime="2026-10-30 21:33:11.524606307 +0000 UTC m=+12.705726666"
Oct 30 21:33:12.411310 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:12.411236    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dgbt5" event={"ID":"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1","Type":"ContainerStarted","Data":"42a108bfdbdb141ca40344b18c824754927933c6a9dbd27234f9e877cfa9a653"}
Oct 30 21:33:12.413864 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:12.413832    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b" event={"ID":"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f","Type":"ContainerStarted","Data":"e24329bcaa67f3c697d6d1b53d0f475c0e1050b34f8be9e93e1613c2061dfe9c"}
Oct 30 21:33:12.416627 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:12.416601    2586 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="1390a7d37fe5cc41f320afaa8ecacb0a3017d6f24f9c96572b9f2e43e13c6d9f" exitCode=0
Oct 30 21:33:12.417523 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:12.417499    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerDied","Data":"1390a7d37fe5cc41f320afaa8ecacb0a3017d6f24f9c96572b9f2e43e13c6d9f"}
Oct 30 21:33:12.526946 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:12.525872    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-svcjf" podStartSLOduration=3.729832538 podStartE2EDuration="12.525854004s" totalImagesPullingTime="8.796021466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:11.774886914 +0000 UTC m=+12.956007270" watchObservedRunningTime="2026-10-30 21:33:12.525854004 +0000 UTC m=+13.706974361"
Oct 30 21:33:12.526946 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:12.526401    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dgbt5" podStartSLOduration=3.357488836 podStartE2EDuration="12.526388156s" totalImagesPullingTime="9.16889932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:12.526194667 +0000 UTC m=+13.707315012" watchObservedRunningTime="2026-10-30 21:33:12.526388156 +0000 UTC m=+13.707508513"
Oct 30 21:33:12.644314 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:12.644265    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b" podStartSLOduration=2.680131304 podStartE2EDuration="12.644255713s" totalImagesPullingTime="9.964124409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:12.643745144 +0000 UTC m=+13.824865498" watchObservedRunningTime="2026-10-30 21:33:12.644255713 +0000 UTC m=+13.825376066"
Oct 30 21:33:13.282714 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:13.282639    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:13.282714 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:13.282686    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:13.282892 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:13.282779    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:13.283084 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:13.283066    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:13.283134 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:13.283124    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:13.283188 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:13.283176    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:13.420619 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:13.420583    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24q2x" event={"ID":"b66918c1-6a0a-495b-bb2e-df4e61590fe4","Type":"ContainerStarted","Data":"a1a696b6d48e9d665dc42863eada6e623f8c3ee3e4f3fd56db9172bc7aa2188c"}
Oct 30 21:33:13.533926 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:13.533830    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-24q2x" podStartSLOduration=2.15083236 podStartE2EDuration="13.533815815s" totalImagesPullingTime="11.382983455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:13.533090921 +0000 UTC m=+14.714211276" watchObservedRunningTime="2026-10-30 21:33:13.533815815 +0000 UTC m=+14.714936169"
Oct 30 21:33:14.424920 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:14.424882    2586 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="b8587a80e6905c668304c4ef2a36081b5dfc0341219ece0c1c1f2508b65c8cd3" exitCode=0
Oct 30 21:33:14.425552 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:14.424960    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerDied","Data":"b8587a80e6905c668304c4ef2a36081b5dfc0341219ece0c1c1f2508b65c8cd3"}
Oct 30 21:33:15.283089 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:15.283054    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:15.283232 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:15.283083    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:15.283232 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:15.283168    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:15.283232 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:15.283182    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:15.283386 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:15.283259    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:15.283386 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:15.283325    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:16.432304 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:16.432275    2586 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="d9f0898e0fb066f0e8a7ba7ffee05f0ad758e61d853914d44ff15bc8f85f4589" exitCode=0
Oct 30 21:33:16.432923 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:16.432322    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerDied","Data":"d9f0898e0fb066f0e8a7ba7ffee05f0ad758e61d853914d44ff15bc8f85f4589"}
Oct 30 21:33:17.111447 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.111251    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:17.111595 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.111458    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:17.111595 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.111408    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:17.111595 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.111546    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:17.111595 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.111562    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs podName:8953ad86-e744-4251-99cc-278bc81abe74 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:33.111547157 +0000 UTC m=+34.292667489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs") pod "network-metrics-daemon-t6n8r" (UID: "8953ad86-e744-4251-99cc-278bc81abe74") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:17.111595 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.111583    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret podName:1a7bdebe-605a-432d-8cc7-91fc5417158d nodeName:}" failed. No retries permitted until 2026-10-30 21:33:33.111571647 +0000 UTC m=+34.292691978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret") pod "global-pull-secret-syncer-2glxt" (UID: "1a7bdebe-605a-432d-8cc7-91fc5417158d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:17.212407 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.212380    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:17.212558 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.212476    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:17.212558 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.212487    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:17.212558 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.212497    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8q5kq for pod openshift-network-diagnostics/network-check-target-b6gtf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:17.212558 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.212535    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq podName:d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:33.212523874 +0000 UTC m=+34.393644205 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q5kq" (UniqueName: "kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq") pod "network-check-target-b6gtf" (UID: "d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:17.282730 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.282705    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:17.282821 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.282802    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:17.282821 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.282705    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:17.282890 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.282840    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:17.282959 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.282937    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:17.283046 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:17.283029    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:17.436424 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.436246    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:33:17.437312 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.437285    2586 generic.go:356] "Generic (PLEG): container finished" podID="e0ceab3b-942e-4950-bb5e-96f62e431365" containerID="967a8f20482dc3c186c80be22f12d77899f50731e11603f5dca2f853ae5c091f" exitCode=1
Oct 30 21:33:17.437426 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.437338    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerDied","Data":"967a8f20482dc3c186c80be22f12d77899f50731e11603f5dca2f853ae5c091f"}
Oct 30 21:33:17.437485 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:17.437434    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"9a87afc0b483a7c5b04a32c0c8490045fc89987f83d29122dec415041f7fcc35"}
Oct 30 21:33:18.443095 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:18.442918    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:33:18.443561 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:18.443536    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"6fb2301404ba4594a05be673e9ed68dafe3b1ac7a67ac23aaac8dfad68a89143"}
Oct 30 21:33:18.443633 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:18.443604    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"46ac513f837519978da030f9d94e8ee00be80ca4a86e3a5a124adbb7cccf7601"}
Oct 30 21:33:18.443699 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:18.443651    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"a68f35bc1a068583ddd5fdda5dced074cb91f417a68b6bcc8c5c62135351ad84"}
Oct 30 21:33:18.443699 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:18.443682    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"5a938e874d3e4f7a39171a7f710984852bf51e5d048cc2dd55996e06e55ba80a"}
Oct 30 21:33:18.446376 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:18.446337    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerStarted","Data":"d3874591606539dc333a77e64a74f133466c18e0c165c18cf2209d8dd6d4d3d4"}
Oct 30 21:33:19.283572 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:19.283544    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:19.283702 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:19.283667    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:19.283702 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:19.283671    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:19.283847 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:19.283829    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:19.283886 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:19.283857    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:19.283949 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:19.283937    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:19.450704 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:19.450677    2586 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="d3874591606539dc333a77e64a74f133466c18e0c165c18cf2209d8dd6d4d3d4" exitCode=0
Oct 30 21:33:19.450704 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:19.450713    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerDied","Data":"d3874591606539dc333a77e64a74f133466c18e0c165c18cf2209d8dd6d4d3d4"}
Oct 30 21:33:20.455756 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:20.455727    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:33:20.456134 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:20.456068    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"09960211bdb2dd33e19fe2831b4f3ea984038536ec2e1d3adc9c55c1f4c413d4"}
Oct 30 21:33:20.458938 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:20.458913    2586 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="44ff3699a7e23e811446fd1dc3d9ffb5646b526ca02140e2729b937301a27963" exitCode=0
Oct 30 21:33:20.459070 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:20.458944    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerDied","Data":"44ff3699a7e23e811446fd1dc3d9ffb5646b526ca02140e2729b937301a27963"}
Oct 30 21:33:21.282634 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:21.282581    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:21.282634 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:21.282627    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:21.282841 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:21.282711    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:21.282841 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:21.282750    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:21.282841 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:21.282834    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:21.282958 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:21.282932    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:21.463913 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:21.463882    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" event={"ID":"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15","Type":"ContainerStarted","Data":"daaedb5d538e8a05f185e9fa862454ac670ca27925b13504cf5695f2d5f942d9"}
Oct 30 21:33:21.634152 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:21.632757    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s5wqs" podStartSLOduration=9.108098682 podStartE2EDuration="21.63273386s" totalImagesPullingTime="12.524635178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:21.631892107 +0000 UTC m=+22.813012462" watchObservedRunningTime="2026-10-30 21:33:21.63273386 +0000 UTC m=+22.813854215"
Oct 30 21:33:22.469609 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:22.469583    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:33:22.470200 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:22.470006    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"c960fcbb9ccfa8900d2dc19f8b71d3f95befac0656ce1168fc5dfcae7be4eabe"}
Oct 30 21:33:22.470533 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:22.470517    2586 scope.go:122] "RemoveContainer" containerID="967a8f20482dc3c186c80be22f12d77899f50731e11603f5dca2f853ae5c091f"
Oct 30 21:33:23.282158 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.282128    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:23.282158 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.282148    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:23.282491 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:23.282257    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:23.282491 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:23.282320    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:23.282491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.282377    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:23.282491 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:23.282454    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:23.476275 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.476245    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:33:23.476713 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.476629    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" event={"ID":"e0ceab3b-942e-4950-bb5e-96f62e431365","Type":"ContainerStarted","Data":"d5e4e48a7cc306adf377aacf348fe0ee2511b7cb4b93a7d9aad1513d11033fc4"}
Oct 30 21:33:23.476768 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.476716    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:33:23.476976 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.476942    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:23.477109 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.477008    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:23.491660 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.491640    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:23.491763 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.491730    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:23.655691 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.655625    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:23.679343 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:23.679298    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh" podStartSLOduration=8.105209934 podStartE2EDuration="23.679287453s" totalImagesPullingTime="15.574077519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:23.646513162 +0000 UTC m=+24.827633515" watchObservedRunningTime="2026-10-30 21:33:23.679287453 +0000 UTC m=+24.860407801"
Oct 30 21:33:24.456843 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:24.456770    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2glxt"]
Oct 30 21:33:24.457001 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:24.456934    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:24.457056 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:24.457039    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:24.460301 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:24.460099    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6gtf"]
Oct 30 21:33:24.460301 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:24.460199    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:24.460301 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:24.460269    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:24.460970 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:24.460940    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t6n8r"]
Oct 30 21:33:24.461072 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:24.461031    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:24.461137 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:24.461120    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:26.052416 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:26.052340    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:26.052851 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:26.052470    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:33:26.052888 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:26.052872    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:33:26.282997 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:26.282970    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:26.283173 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:26.283054    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:26.283173 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:26.283147    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:26.283291 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:26.283268    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:26.283328 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:26.283300    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:26.283396 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:26.283382    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:26.327896 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:26.327841    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zmw2g_e6157882-1c00-4bb2-9790-25ba55f8ceb7/dns-node-resolver/0.log"
Oct 30 21:33:27.163525 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:27.163499    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sswwd_19c8fb74-659e-4087-a035-3f26fa17b33c/node-ca/0.log"
Oct 30 21:33:28.282459 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:28.282429    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:28.282459 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:28.282451    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:28.282820 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:28.282542    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:28.282820 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:28.282594    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:28.282820 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:28.282675    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:28.282820 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:28.282739    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:30.282371 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:30.282282    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:30.282753 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:30.282400    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:30.282753 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:30.282421    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:30.282753 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:30.282400    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:30.282753 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:30.282515    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:30.282753 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:30.282578    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:32.282527 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:32.282487    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:32.282527 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:32.282529    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:32.282932 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:32.282589    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:32.282932 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:32.282600    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:32.282932 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:32.282661    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:32.282932 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:32.282731    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:33.125597 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:33.125568    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:33.125597 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:33.125601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:33.125804 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.125693    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:33.125804 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.125699    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:33.125804 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.125737    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret podName:1a7bdebe-605a-432d-8cc7-91fc5417158d nodeName:}" failed. No retries permitted until 2026-10-30 21:34:05.125724774 +0000 UTC m=+66.306845106 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret") pod "global-pull-secret-syncer-2glxt" (UID: "1a7bdebe-605a-432d-8cc7-91fc5417158d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:33.125804 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.125750    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs podName:8953ad86-e744-4251-99cc-278bc81abe74 nodeName:}" failed. No retries permitted until 2026-10-30 21:34:05.125744116 +0000 UTC m=+66.306864448 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs") pod "network-metrics-daemon-t6n8r" (UID: "8953ad86-e744-4251-99cc-278bc81abe74") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:33.225865 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:33.225841    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:33.226112 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.225944    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:33.226112 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.225960    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:33.226112 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.225972    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8q5kq for pod openshift-network-diagnostics/network-check-target-b6gtf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:33.226112 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:33.226025    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq podName:d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10 nodeName:}" failed. No retries permitted until 2026-10-30 21:34:05.226009619 +0000 UTC m=+66.407129966 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q5kq" (UniqueName: "kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq") pod "network-check-target-b6gtf" (UID: "d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:34.282321 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:34.282298    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:34.282674 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:34.282299    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:34.282674 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:34.282407    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:34.282674 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:34.282302    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:34.282674 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:34.282464    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:34.282674 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:34.282541    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:36.283243 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:36.283212    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:36.283243 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:36.283241    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:36.283674 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:36.283322    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:36.283674 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:36.283376    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:36.283674 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:36.283458    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:36.283674 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:36.283507    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:38.282960 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:38.282929    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:38.283335 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:38.282929    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:38.283335 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:38.283025    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:38.283335 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:38.282930    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:38.283335 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:38.283101    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:38.283335 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:38.283204    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:40.282647 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:40.282613    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:40.282647 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:40.282648    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:40.283178 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:40.282652    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:40.283178 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:40.282734    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:40.283178 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:40.282765    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:40.283178 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:40.282830    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:42.282580 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:42.282554    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:42.282964 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:42.282554    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:42.282964 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:42.282654    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:42.282964 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:42.282554    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:42.282964 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:42.282741    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:42.282964 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:42.282802    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:44.282684 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:44.282660    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:44.283047 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:44.282659    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:44.283047 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:44.282757    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:44.283047 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:44.282659    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:44.283047 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:44.282881    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:44.283047 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:44.282813    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:46.282471 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.282440    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:46.282471 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.282468    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:46.282842 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:46.282541    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6n8r" podUID="8953ad86-e744-4251-99cc-278bc81abe74"
Oct 30 21:33:46.282842 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.282594    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:46.282842 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:46.282667    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2glxt" podUID="1a7bdebe-605a-432d-8cc7-91fc5417158d"
Oct 30 21:33:46.282842 ip-10-0-135-39 kubenswrapper[2586]: E1030 21:33:46.282715    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6gtf" podUID="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10"
Oct 30 21:33:46.594642 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.594580    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:33:46.594765 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.594667    2586 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:33:46.757985 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.757950    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q94hx"]
Oct 30 21:33:46.759551 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.759536    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tvrlz"]
Oct 30 21:33:46.759672 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.759655    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:46.761221 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.761203    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:46.764753 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.764733    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q94hx"]
Oct 30 21:33:46.808551 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.808533    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pzsfj"]
Oct 30 21:33:46.809983 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.809968    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:46.811469 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.811452    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pczpt\""
Oct 30 21:33:46.811564 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.811488    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:33:46.815254 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.815233    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:33:46.815370 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.815310    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:33:46.815419 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.815371    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:33:46.815419 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.815401    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:33:46.832563 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.832543    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tvrlz"]
Oct 30 21:33:46.870204 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.870122    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-bfb94f7fb-rc2vj"]
Oct 30 21:33:46.871627 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.871611    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:46.873311 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.873292    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pzsfj"]
Oct 30 21:33:46.912034 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.912010    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:33:46.912159 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.912145    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-flltm\""
Oct 30 21:33:46.912159 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.912156    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26jx5\""
Oct 30 21:33:46.912259 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.912188    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:33:46.916138 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.916124    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:33:46.916188 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.916171    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:33:46.923211 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923197    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:33:46.923440 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923424    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5hnkf\""
Oct 30 21:33:46.923523 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923468    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:33:46.923523 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923490    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:33:46.923523 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923516    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:33:46.923635 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923527    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:33:46.923635 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923544    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:33:46.923635 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.923554    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:33:46.933573 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933552    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95784d9c-a51b-4836-b072-b2c12568392e-data-volume\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:46.933668 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933580    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7a665f-7b9e-415b-8d52-b355e803032d-metrics-tls\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:46.933668 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933603    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95784d9c-a51b-4836-b072-b2c12568392e-crio-socket\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:46.933668 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933641    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7fgn\" (UniqueName: \"kubernetes.io/projected/574060d7-1710-4556-8f71-347eac712504-kube-api-access-k7fgn\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:46.933813 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933739    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95784d9c-a51b-4836-b072-b2c12568392e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:46.933813 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933759    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r47w2\" (UniqueName: \"kubernetes.io/projected/95784d9c-a51b-4836-b072-b2c12568392e-kube-api-access-r47w2\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:46.933813 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933775    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a665f-7b9e-415b-8d52-b355e803032d-config-volume\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:46.933813 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933794    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb2b2\" (UniqueName: \"kubernetes.io/projected/2f7a665f-7b9e-415b-8d52-b355e803032d-kube-api-access-lb2b2\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:46.933987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933831    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/574060d7-1710-4556-8f71-347eac712504-cert\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:46.933987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933880    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95784d9c-a51b-4836-b072-b2c12568392e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:46.933987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.933933    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f7a665f-7b9e-415b-8d52-b355e803032d-tmp-dir\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:46.972866 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.972848    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:33:46.973564 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:46.973536    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bfb94f7fb-rc2vj"]
Oct 30 21:33:47.034202 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034177    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7fgn\" (UniqueName: \"kubernetes.io/projected/574060d7-1710-4556-8f71-347eac712504-kube-api-access-k7fgn\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:47.034290 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034239    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034290 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034271    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95784d9c-a51b-4836-b072-b2c12568392e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.034290 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034288    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r47w2\" (UniqueName: \"kubernetes.io/projected/95784d9c-a51b-4836-b072-b2c12568392e-kube-api-access-r47w2\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.034453 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034389    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a665f-7b9e-415b-8d52-b355e803032d-config-volume\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.034453 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034427    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb2b2\" (UniqueName: \"kubernetes.io/projected/2f7a665f-7b9e-415b-8d52-b355e803032d-kube-api-access-lb2b2\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.034550 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034457    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/574060d7-1710-4556-8f71-347eac712504-cert\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:47.034599 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034584    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95784d9c-a51b-4836-b072-b2c12568392e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.034652 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034608    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034652 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034637    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034749 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034659    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034806 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034782    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f7a665f-7b9e-415b-8d52-b355e803032d-tmp-dir\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.034872 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034812    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034872 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034832    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2z7p\" (UniqueName: \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034958 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034871    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034958 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034930    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.034958 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034951    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95784d9c-a51b-4836-b072-b2c12568392e-data-volume\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.035121 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034966    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7a665f-7b9e-415b-8d52-b355e803032d-metrics-tls\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.035121 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.034992    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95784d9c-a51b-4836-b072-b2c12568392e-crio-socket\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.035121 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.035057    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f7a665f-7b9e-415b-8d52-b355e803032d-tmp-dir\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.035121 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.035081    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a665f-7b9e-415b-8d52-b355e803032d-config-volume\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.035277 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.035135    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95784d9c-a51b-4836-b072-b2c12568392e-crio-socket\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.035277 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.035187    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95784d9c-a51b-4836-b072-b2c12568392e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.035378 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.035277    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95784d9c-a51b-4836-b072-b2c12568392e-data-volume\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.038055 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.038028    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95784d9c-a51b-4836-b072-b2c12568392e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.038144 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.038032    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7a665f-7b9e-415b-8d52-b355e803032d-metrics-tls\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.038144 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.038074    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/574060d7-1710-4556-8f71-347eac712504-cert\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:47.089937 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.089907    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb2b2\" (UniqueName: \"kubernetes.io/projected/2f7a665f-7b9e-415b-8d52-b355e803032d-kube-api-access-lb2b2\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.090336 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.090312    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7fgn\" (UniqueName: \"kubernetes.io/projected/574060d7-1710-4556-8f71-347eac712504-kube-api-access-k7fgn\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:47.090592 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.090575    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r47w2\" (UniqueName: \"kubernetes.io/projected/95784d9c-a51b-4836-b072-b2c12568392e-kube-api-access-r47w2\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.118653 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.118635    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:33:47.136182 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136163    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136273 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136197    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136273 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136239    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136273 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136260    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136453 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136407    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136453 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136445    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136552 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136478    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136552 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136508    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2z7p\" (UniqueName: \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.136941 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.136881    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.137026 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.137003    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.137026 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.137019    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.137319 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.137301    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.137440 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.137424    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.138830 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.138809    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.138946 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.138927    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.193560 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.193530    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2z7p\" (UniqueName: \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.374286 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.374222    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:33:47.378535 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.378513    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:47.484467 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.484438    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:47.490744 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.490724    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pzsfj"]
Oct 30 21:33:47.494545 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:47.494523    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574060d7_1710_4556_8f71_347eac712504.slice/crio-7ea26db45ccc7993a35faed6e41c0e4c02da4e4978f3e677ef60520cab2b2c7d WatchSource:0}: Error finding container 7ea26db45ccc7993a35faed6e41c0e4c02da4e4978f3e677ef60520cab2b2c7d: Status 404 returned error can't find the container with id 7ea26db45ccc7993a35faed6e41c0e4c02da4e4978f3e677ef60520cab2b2c7d
Oct 30 21:33:47.521890 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.521862    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pzsfj" event={"ID":"574060d7-1710-4556-8f71-347eac712504","Type":"ContainerStarted","Data":"7ea26db45ccc7993a35faed6e41c0e4c02da4e4978f3e677ef60520cab2b2c7d"}
Oct 30 21:33:47.737662 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.737636    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q94hx"]
Oct 30 21:33:47.740207 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:47.740176    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95784d9c_a51b_4836_b072_b2c12568392e.slice/crio-a3e9d03a03e2ddb5f06770708776619d2e85fd210eb6a4aeade3e6e34e592c75 WatchSource:0}: Error finding container a3e9d03a03e2ddb5f06770708776619d2e85fd210eb6a4aeade3e6e34e592c75: Status 404 returned error can't find the container with id a3e9d03a03e2ddb5f06770708776619d2e85fd210eb6a4aeade3e6e34e592c75
Oct 30 21:33:47.786129 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.785893    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tvrlz"]
Oct 30 21:33:47.789317 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:47.789293    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f7a665f_7b9e_415b_8d52_b355e803032d.slice/crio-c1fc6e24b8f47ca8e605eb9f4e9604d14428ec9d240f3341b93dc35ec4c0dc42 WatchSource:0}: Error finding container c1fc6e24b8f47ca8e605eb9f4e9604d14428ec9d240f3341b93dc35ec4c0dc42: Status 404 returned error can't find the container with id c1fc6e24b8f47ca8e605eb9f4e9604d14428ec9d240f3341b93dc35ec4c0dc42
Oct 30 21:33:47.848445 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:47.848382    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bfb94f7fb-rc2vj"]
Oct 30 21:33:47.851514 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:47.851488    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170fa4df_5ce6_4243_b30a_7b2c91a733ce.slice/crio-ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1 WatchSource:0}: Error finding container ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1: Status 404 returned error can't find the container with id ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1
Oct 30 21:33:48.282903 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.282878    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:33:48.283068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.282878    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:33:48.283068 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.282996    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:33:48.334763 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.334737    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:33:48.334763 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.334743    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:33:48.334988 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.334972    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76thg\""
Oct 30 21:33:48.335058 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.335042    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:33:48.335096 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.335061    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-cprz9\""
Oct 30 21:33:48.335096 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.335041    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qlt7z\""
Oct 30 21:33:48.335157 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.335101    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:33:48.525031 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.524994    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb94f7fb-rc2vj" event={"ID":"170fa4df-5ce6-4243-b30a-7b2c91a733ce","Type":"ContainerStarted","Data":"ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1"}
Oct 30 21:33:48.526085 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.526065    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tvrlz" event={"ID":"2f7a665f-7b9e-415b-8d52-b355e803032d","Type":"ContainerStarted","Data":"c1fc6e24b8f47ca8e605eb9f4e9604d14428ec9d240f3341b93dc35ec4c0dc42"}
Oct 30 21:33:48.530736 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.530710    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q94hx" event={"ID":"95784d9c-a51b-4836-b072-b2c12568392e","Type":"ContainerStarted","Data":"9e254f8c43207232fced5b605bf10d085318934597abb163a3b3188a985ed6c3"}
Oct 30 21:33:48.530817 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:48.530775    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q94hx" event={"ID":"95784d9c-a51b-4836-b072-b2c12568392e","Type":"ContainerStarted","Data":"a3e9d03a03e2ddb5f06770708776619d2e85fd210eb6a4aeade3e6e34e592c75"}
Oct 30 21:33:49.536158 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:49.536111    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q94hx" event={"ID":"95784d9c-a51b-4836-b072-b2c12568392e","Type":"ContainerStarted","Data":"8e929017fa068d042878bbcae1a9e6336bf1ac3e6d954f0da5019d51822099dc"}
Oct 30 21:33:50.404312 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.403374    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"]
Oct 30 21:33:50.405387 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.405316    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.458024 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.457994    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 21:33:50.458159 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.458146    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:33:50.458266 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.458243    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 21:33:50.458424 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.458367    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:33:50.458516 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.458479    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k7slv\""
Oct 30 21:33:50.458578 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.458538    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:33:50.464981 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.464950    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"]
Oct 30 21:33:50.543277 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.543244    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tvrlz" event={"ID":"2f7a665f-7b9e-415b-8d52-b355e803032d","Type":"ContainerStarted","Data":"74b62b45131ccd376abe44021d05f32d8ad718c5e3c110dd1ff7842c503446b9"}
Oct 30 21:33:50.543673 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.543306    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tvrlz" event={"ID":"2f7a665f-7b9e-415b-8d52-b355e803032d","Type":"ContainerStarted","Data":"0e64b89327a9e100c3374a0967fe0294dbd56a500750f8b4d741930c52c3821d"}
Oct 30 21:33:50.544076 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.544049    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:33:50.562697 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.562670    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.562799 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.562730    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2rfm\" (UniqueName: \"kubernetes.io/projected/c7eabbd5-c926-4eaf-9629-17b0da3ec706-kube-api-access-g2rfm\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.562850 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.562794    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.562888 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.562863    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7eabbd5-c926-4eaf-9629-17b0da3ec706-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.664942 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.664861    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2rfm\" (UniqueName: \"kubernetes.io/projected/c7eabbd5-c926-4eaf-9629-17b0da3ec706-kube-api-access-g2rfm\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.664942 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.664918    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.665157 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.664987    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7eabbd5-c926-4eaf-9629-17b0da3ec706-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.665157 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.664961    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-tvrlz" podStartSLOduration=2.432401726 podStartE2EDuration="4.664943262s" totalImagesPullingTime="2.232541536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:50.660742792 +0000 UTC m=+51.841863138" watchObservedRunningTime="2026-10-30 21:33:50.664943262 +0000 UTC m=+51.846063617"
Oct 30 21:33:50.665157 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.665133    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.667188 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.667144    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7eabbd5-c926-4eaf-9629-17b0da3ec706-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.668977 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.668957    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.669841 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.669816    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:50.722411 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:50.722383    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2rfm\" (UniqueName: \"kubernetes.io/projected/c7eabbd5-c926-4eaf-9629-17b0da3ec706-kube-api-access-g2rfm\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:51.020390 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:51.020299    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:33:51.393923 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:51.393882    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"]
Oct 30 21:33:51.548200 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:51.548173    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q94hx" event={"ID":"95784d9c-a51b-4836-b072-b2c12568392e","Type":"ContainerStarted","Data":"6bbbfc9c2daaff3b201096646dda75eac5e92d7d78218bc99d127ccf8448417f"}
Oct 30 21:33:51.549402 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:51.549377    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb" event={"ID":"c7eabbd5-c926-4eaf-9629-17b0da3ec706","Type":"ContainerStarted","Data":"1675e3e8eeb657d4413af16c63aa25191ad5b93a36907b74a28f07571872b869"}
Oct 30 21:33:51.662484 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:51.662424    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q94hx" podStartSLOduration=2.278656531 podStartE2EDuration="5.662403941s" totalImagesPullingTime="3.38374741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:51.662120094 +0000 UTC m=+52.843240488" watchObservedRunningTime="2026-10-30 21:33:51.662403941 +0000 UTC m=+52.843524297"
Oct 30 21:33:52.554134 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:52.554100    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb94f7fb-rc2vj" event={"ID":"170fa4df-5ce6-4243-b30a-7b2c91a733ce","Type":"ContainerStarted","Data":"ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94"}
Oct 30 21:33:52.555707 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:52.555681    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pzsfj" event={"ID":"574060d7-1710-4556-8f71-347eac712504","Type":"ContainerStarted","Data":"b92902c73077b81c932b8c785e7b53bc57a50e49eeab56231088c63f5962e7b0"}
Oct 30 21:33:52.720040 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:52.719990    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-bfb94f7fb-rc2vj" podStartSLOduration=3.044117199 podStartE2EDuration="6.719975706s" totalImagesPullingTime="3.675858507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:52.71901975 +0000 UTC m=+53.900140101" watchObservedRunningTime="2026-10-30 21:33:52.719975706 +0000 UTC m=+53.901096135"
Oct 30 21:33:52.832364 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:52.832246    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pzsfj" podStartSLOduration=1.830011941 podStartE2EDuration="6.832231089s" totalImagesPullingTime="5.002219148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:52.83163205 +0000 UTC m=+54.012752403" watchObservedRunningTime="2026-10-30 21:33:52.832231089 +0000 UTC m=+54.013351453"
Oct 30 21:33:53.560557 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:53.560470    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb" event={"ID":"c7eabbd5-c926-4eaf-9629-17b0da3ec706","Type":"ContainerStarted","Data":"27992530d2d855436ad2f3acb97ca2d1e8130c6d7f09f27b7c88ff7d5b4f2aa0"}
Oct 30 21:33:53.560557 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:53.560548    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb" event={"ID":"c7eabbd5-c926-4eaf-9629-17b0da3ec706","Type":"ContainerStarted","Data":"5d4a3c89345d74e7230996f2f1277a2ebab754fb0fe863832399a8909d1f2471"}
Oct 30 21:33:53.685153 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:53.685107    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb" podStartSLOduration=1.882232722 podStartE2EDuration="3.68509457s" totalImagesPullingTime="1.802861848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:53.677921246 +0000 UTC m=+54.859041600" watchObservedRunningTime="2026-10-30 21:33:53.68509457 +0000 UTC m=+54.866214924"
Oct 30 21:33:55.491153 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:55.491125    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:33:56.959845 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:56.959815    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zqjjz"]
Oct 30 21:33:56.961692 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:56.961676    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.010740 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.010716    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-root\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.010859 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.010747    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-metrics-client-ca\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.010859 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.010767    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25d5t\" (UniqueName: \"kubernetes.io/projected/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-kube-api-access-25d5t\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.010859 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.010784    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.010859 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.010810    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-wtmp\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.011002 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.010940    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-sys\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.011002 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.010972    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-textfile\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.011105 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.011043    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.011105 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.011072    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-tls\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.018452 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.018430    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:33:57.018615 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.018436    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:33:57.067588 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.067564    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:33:57.067689 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.067591    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2n6f\""
Oct 30 21:33:57.112193 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112168    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-tls\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112302 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112197    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-root\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112302 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112215    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-metrics-client-ca\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112302 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112242    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25d5t\" (UniqueName: \"kubernetes.io/projected/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-kube-api-access-25d5t\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112302 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112261    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112302 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112288    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-wtmp\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112302 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112292    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-root\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112640 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112343    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-sys\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112640 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112395    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-textfile\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112640 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112441    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112640 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112451    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-wtmp\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112640 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112466    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-sys\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112889 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112734    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-textfile\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.112940 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.112917    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-metrics-client-ca\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.113121 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.113101    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.114666 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.114648    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-tls\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.114763 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.114703    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.167996 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.167967    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25d5t\" (UniqueName: \"kubernetes.io/projected/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-kube-api-access-25d5t\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.274756 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.274701    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:33:57.283999 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:33:57.283977    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be22202_fcae_4c90_b4ff_0eb07a3c41e0.slice/crio-baa186044f9912f0174d6c85ef3310d50ee4ea462c2f4e75e2da0b85faaa982f WatchSource:0}: Error finding container baa186044f9912f0174d6c85ef3310d50ee4ea462c2f4e75e2da0b85faaa982f: Status 404 returned error can't find the container with id baa186044f9912f0174d6c85ef3310d50ee4ea462c2f4e75e2da0b85faaa982f
Oct 30 21:33:57.485389 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.485339    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:57.485508 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.485419    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:57.490192 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.490168    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:57.573678 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.573604    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zqjjz" event={"ID":"2be22202-fcae-4c90-b4ff-0eb07a3c41e0","Type":"ContainerStarted","Data":"baa186044f9912f0174d6c85ef3310d50ee4ea462c2f4e75e2da0b85faaa982f"}
Oct 30 21:33:57.577624 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:57.577604    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:33:59.293491 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.293460    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:33:59.295801 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.295779    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.326879 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.326855    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.326981 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.326907    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327021 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.326979    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327021 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327016    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327082 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327037    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327114 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327088    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327146 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327120    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327202 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327188    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327235 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327214    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsw5v\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327266 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327247    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327297 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327277    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327379 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327365    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.327417 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.327406    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.350969 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.350936    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:33:59.352591 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.352571    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:33:59.400912 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.400887    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:33:59.400912 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.400889    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:33:59.401081 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.400940    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:33:59.427878 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.427854    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.427974 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.427881    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.427974 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.427903    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.427974 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.427920    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.427974 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.427962    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.427981    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.427997    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428014    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428043    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428067    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsw5v\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428095    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428164 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428129    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428516 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428305    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428572 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428517    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.428936 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.428905    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.430605 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.430587    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.430691 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.430621    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.430758 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.430694    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.430865 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.430845    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.431031 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.431014    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.450063 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.450045    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:33:59.450614 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.450598    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:33:59.450685 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.450663    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7hq5v\""
Oct 30 21:33:59.460743 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.460725    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.460857 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.460842    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.499765 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.499747    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:33:59.499765 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.499759    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:33:59.499916 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.499888    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:33:59.501788 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.501769    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsw5v\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.509168 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.509148    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.510514 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.510493    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.510514 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.510508    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:33:59.581264 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.581213    2586 generic.go:356] "Generic (PLEG): container finished" podID="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" containerID="4c4b82d5e61e6aaffa87d7188e58abd4120a320dc9d619ee812a1b7dede6d6a8" exitCode=0
Oct 30 21:33:59.581264 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.581242    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zqjjz" event={"ID":"2be22202-fcae-4c90-b4ff-0eb07a3c41e0","Type":"ContainerDied","Data":"4c4b82d5e61e6aaffa87d7188e58abd4120a320dc9d619ee812a1b7dede6d6a8"}
Oct 30 21:33:59.609987 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:33:59.609957    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:00.033280 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:00.033245    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:34:00.036481 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:34:00.036451    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6b80f7_f5f7_4a39_adb2_2e4ff8e02d94.slice/crio-5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc WatchSource:0}: Error finding container 5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc: Status 404 returned error can't find the container with id 5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc
Oct 30 21:34:00.586303 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:00.586268    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zqjjz" event={"ID":"2be22202-fcae-4c90-b4ff-0eb07a3c41e0","Type":"ContainerStarted","Data":"e60fbdb6eeed3c6803ee471f86ffe2577747fe3a8a5174e292360104d93cfdee"}
Oct 30 21:34:00.586729 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:00.586378    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zqjjz" event={"ID":"2be22202-fcae-4c90-b4ff-0eb07a3c41e0","Type":"ContainerStarted","Data":"816f0632100ffe74c4e0b08a115fae577be6207d217c4801194866146784a5ea"}
Oct 30 21:34:00.587447 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:00.587426    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerStarted","Data":"5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc"}
Oct 30 21:34:00.701222 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:00.701178    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zqjjz" podStartSLOduration=3.360781472 podStartE2EDuration="4.70116693s" totalImagesPullingTime="1.274581174s" totalInitContainerRuntime="65.804284ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:00.699753144 +0000 UTC m=+61.880873498" watchObservedRunningTime="2026-10-30 21:34:00.70116693 +0000 UTC m=+61.882287298"
Oct 30 21:34:01.558099 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.558079    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:01.592364 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.592320    2586 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c" exitCode=0
Oct 30 21:34:01.592689 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.592381    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c"}
Oct 30 21:34:01.782075 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.781999    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-66b4c7697-8m46d"]
Oct 30 21:34:01.783729 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.783714    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.836099 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.836063    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 21:34:01.836099 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.836087    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 21:34:01.836099 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.836087    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-f6ntr\""
Oct 30 21:34:01.836396 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.836148    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:34:01.836396 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.836263    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 21:34:01.836475 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.836395    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-alkj5vgb4qp18\""
Oct 30 21:34:01.843847 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.843825    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66b4c7697-8m46d"]
Oct 30 21:34:01.848022 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.848001    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-client-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.848132 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.848033    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-metrics-server-audit-profiles\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.848132 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.848071    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6990798-b558-45b4-86d2-ab0917893407-audit-log\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.848237 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.848161    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-tls\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.848237 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.848184    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.848237 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.848209    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-client-certs\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.848237 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.848224    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w77mn\" (UniqueName: \"kubernetes.io/projected/f6990798-b558-45b4-86d2-ab0917893407-kube-api-access-w77mn\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.948723 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.948677    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-client-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.948892 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.948731    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-metrics-server-audit-profiles\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.948892 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.948781    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6990798-b558-45b4-86d2-ab0917893407-audit-log\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.948892 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.948812    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-tls\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.948892 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.948834    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.948892 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.948865    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-client-certs\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.949140 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.948886    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77mn\" (UniqueName: \"kubernetes.io/projected/f6990798-b558-45b4-86d2-ab0917893407-kube-api-access-w77mn\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.949317 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.949281    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6990798-b558-45b4-86d2-ab0917893407-audit-log\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.949664 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.949644    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.949906 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.949885    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-metrics-server-audit-profiles\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.951372 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.951333    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-client-certs\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.951595 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.951578    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-client-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:01.951696 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:01.951680    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-tls\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:02.005943 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.005912    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w77mn\" (UniqueName: \"kubernetes.io/projected/f6990798-b558-45b4-86d2-ab0917893407-kube-api-access-w77mn\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:02.093281 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.093200    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:02.506730 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.506428    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66b4c7697-8m46d"]
Oct 30 21:34:02.508961 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:34:02.508927    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6990798_b558_45b4_86d2_ab0917893407.slice/crio-fa32f5a732587d04e8e5c6db93443a15cc22aa31b475b81ce3501983c99cbca9 WatchSource:0}: Error finding container fa32f5a732587d04e8e5c6db93443a15cc22aa31b475b81ce3501983c99cbca9: Status 404 returned error can't find the container with id fa32f5a732587d04e8e5c6db93443a15cc22aa31b475b81ce3501983c99cbca9
Oct 30 21:34:02.602040 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.601996    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d" event={"ID":"f6990798-b558-45b4-86d2-ab0917893407","Type":"ContainerStarted","Data":"fa32f5a732587d04e8e5c6db93443a15cc22aa31b475b81ce3501983c99cbca9"}
Oct 30 21:34:02.667872 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.667825    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"]
Oct 30 21:34:02.669993 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.669973    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.722465 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.722425    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:34:02.722815 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.722793    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:34:02.757075 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757005    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-federate-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.757075 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757049    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.757246 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757072    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.757246 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757149    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5f9n\" (UniqueName: \"kubernetes.io/projected/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-kube-api-access-f5f9n\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.757343 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757248    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-serving-certs-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.757343 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757281    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.757455 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757403    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-metrics-client-ca\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.757455 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.757448    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.771718 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.771695    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:34:02.771849 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.771696    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:34:02.772021 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.771994    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wv6gv\""
Oct 30 21:34:02.772227 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.772211    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:34:02.773026 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.773008    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:34:02.774011 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.773984    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"]
Oct 30 21:34:02.858119 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858085    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-federate-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.858276 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858131    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.858276 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858158    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.858276 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858188    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f9n\" (UniqueName: \"kubernetes.io/projected/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-kube-api-access-f5f9n\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.858276 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858223    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-serving-certs-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.858276 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858249    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.858558 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858305    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-metrics-client-ca\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.858558 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.858342    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.859802 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.859747    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.860172 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.860104    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-metrics-client-ca\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.860259 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.860196    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-serving-certs-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.861821 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.861795    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.862368 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.862329    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-federate-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.862651 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.862632    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.864981 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.864958    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.921451 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.921422    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5f9n\" (UniqueName: \"kubernetes.io/projected/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-kube-api-access-f5f9n\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:02.983511 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:02.983482    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:03.408030 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:03.408008    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"]
Oct 30 21:34:03.410127 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:34:03.410103    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd919e55a_538a_48e6_9fe7_9e7d2bdd0ad7.slice/crio-802552c285f1449d87ce5dda5c178118d47afbd1278607c3a4f7b93b253bfb24 WatchSource:0}: Error finding container 802552c285f1449d87ce5dda5c178118d47afbd1278607c3a4f7b93b253bfb24: Status 404 returned error can't find the container with id 802552c285f1449d87ce5dda5c178118d47afbd1278607c3a4f7b93b253bfb24
Oct 30 21:34:03.606081 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:03.606051    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf" event={"ID":"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7","Type":"ContainerStarted","Data":"802552c285f1449d87ce5dda5c178118d47afbd1278607c3a4f7b93b253bfb24"}
Oct 30 21:34:03.608575 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:03.608553    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerStarted","Data":"bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e"}
Oct 30 21:34:03.608673 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:03.608608    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerStarted","Data":"50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626"}
Oct 30 21:34:03.608673 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:03.608640    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerStarted","Data":"eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91"}
Oct 30 21:34:04.615908 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:04.615877    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerStarted","Data":"c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b"}
Oct 30 21:34:04.616292 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:04.615952    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerStarted","Data":"56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e"}
Oct 30 21:34:05.177547 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.177467    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:05.177710 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.177548    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:34:05.228749 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.228714    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:34:05.228883 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.228801    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:34:05.244862 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.240462    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:34:05.250834 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.250810    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:05.278250 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.278223    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:05.329047 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.329014    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:34:05.389885 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.389860    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:34:05.401683 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.401661    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:05.449204 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.449152    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qlt7z\""
Oct 30 21:34:05.455471 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.455450    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76thg\""
Oct 30 21:34:05.458596 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.458577    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:05.465346 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.465329    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:34:05.620705 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.620669    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d" event={"ID":"f6990798-b558-45b4-86d2-ab0917893407","Type":"ContainerStarted","Data":"77c62efe877f09d142a389f05262fd290e3404f8096fb98e0ec95280e80e72c1"}
Oct 30 21:34:05.624325 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.624303    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerStarted","Data":"1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2"}
Oct 30 21:34:05.736651 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.736560    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d" podStartSLOduration=1.764017741 podStartE2EDuration="4.736546121s" totalImagesPullingTime="2.97252838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:05.736084373 +0000 UTC m=+66.917204727" watchObservedRunningTime="2026-10-30 21:34:05.736546121 +0000 UTC m=+66.917666475"
Oct 30 21:34:05.744150 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.744128    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-cprz9\""
Oct 30 21:34:05.748289 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.748270    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-86bc8c7745-jf6t2"]
Oct 30 21:34:05.751346 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.751331    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.753833 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.753815    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:05.781903 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.781877    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.781993 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.781925    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxxzj\" (UniqueName: \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.781993 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.781944    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.782075 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.782047    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.782108 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.782076    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.782108 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.782096    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.782175 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.782115    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.782209 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.782194    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.848460 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.848432    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2glxt"]
Oct 30 21:34:05.851040 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.851016    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bc8c7745-jf6t2"]
Oct 30 21:34:05.851626 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:34:05.851599    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7bdebe_605a_432d_8cc7_91fc5417158d.slice/crio-295769846d6fe93a27d4a57475bc82f9409cbe3c045d7638591bb2735c10a329 WatchSource:0}: Error finding container 295769846d6fe93a27d4a57475bc82f9409cbe3c045d7638591bb2735c10a329: Status 404 returned error can't find the container with id 295769846d6fe93a27d4a57475bc82f9409cbe3c045d7638591bb2735c10a329
Oct 30 21:34:05.863975 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.863932    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.448824993 podStartE2EDuration="6.863918899s" totalImagesPullingTime="4.404428033s" totalInitContainerRuntime="10.665873ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:05.862841185 +0000 UTC m=+67.043961565" watchObservedRunningTime="2026-10-30 21:34:05.863918899 +0000 UTC m=+67.045039280"
Oct 30 21:34:05.883487 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883466    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxxzj\" (UniqueName: \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.883576 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883494    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.883576 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883514    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.883576 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883530    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.883576 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883546    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.883576 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883567    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.883868 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883604    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.883868 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.883632    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.884287 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.884249    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.884383 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.884290    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.884383 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.884317    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.884494 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.884437    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.884613 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.884592    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.886605 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.886590    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.886649 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.886609    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:05.899170 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.899066    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t6n8r"]
Oct 30 21:34:05.901994 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:34:05.901967    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8953ad86_e744_4251_99cc_278bc81abe74.slice/crio-946c6ac6e2d51e3044aeb1071499ef276fd76f06d76c623accf0366c1a8f6356 WatchSource:0}: Error finding container 946c6ac6e2d51e3044aeb1071499ef276fd76f06d76c623accf0366c1a8f6356: Status 404 returned error can't find the container with id 946c6ac6e2d51e3044aeb1071499ef276fd76f06d76c623accf0366c1a8f6356
Oct 30 21:34:05.945764 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:05.945730    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxxzj\" (UniqueName: \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:06.064458 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.064389    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:06.129272 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.129238    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6gtf"]
Oct 30 21:34:06.132599 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:34:06.132561    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9fa252f_d4f0_4d3b_b2e2_cd9f7cf5fc10.slice/crio-8fb6ffd95f21a764074825c274986126293daa335d28532cc4e0abb479e2be95 WatchSource:0}: Error finding container 8fb6ffd95f21a764074825c274986126293daa335d28532cc4e0abb479e2be95: Status 404 returned error can't find the container with id 8fb6ffd95f21a764074825c274986126293daa335d28532cc4e0abb479e2be95
Oct 30 21:34:06.434891 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.434867    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bc8c7745-jf6t2"]
Oct 30 21:34:06.437511 ip-10-0-135-39 kubenswrapper[2586]: W1030 21:34:06.437483    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93dfe0f_83db_4682_a489_67ca3f81ee3b.slice/crio-f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625 WatchSource:0}: Error finding container f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625: Status 404 returned error can't find the container with id f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625
Oct 30 21:34:06.628549 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.628511    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2glxt" event={"ID":"1a7bdebe-605a-432d-8cc7-91fc5417158d","Type":"ContainerStarted","Data":"295769846d6fe93a27d4a57475bc82f9409cbe3c045d7638591bb2735c10a329"}
Oct 30 21:34:06.629781 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.629759    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6gtf" event={"ID":"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10","Type":"ContainerStarted","Data":"8fb6ffd95f21a764074825c274986126293daa335d28532cc4e0abb479e2be95"}
Oct 30 21:34:06.630895 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.630873    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6n8r" event={"ID":"8953ad86-e744-4251-99cc-278bc81abe74","Type":"ContainerStarted","Data":"946c6ac6e2d51e3044aeb1071499ef276fd76f06d76c623accf0366c1a8f6356"}
Oct 30 21:34:06.632442 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.632412    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc8c7745-jf6t2" event={"ID":"e93dfe0f-83db-4682-a489-67ca3f81ee3b","Type":"ContainerStarted","Data":"ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786"}
Oct 30 21:34:06.632530 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.632460    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc8c7745-jf6t2" event={"ID":"e93dfe0f-83db-4682-a489-67ca3f81ee3b","Type":"ContainerStarted","Data":"f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625"}
Oct 30 21:34:06.746417 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:06.746126    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-86bc8c7745-jf6t2" podStartSLOduration=1.7461096189999998 podStartE2EDuration="1.746109619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:06.745710739 +0000 UTC m=+67.926831093" watchObservedRunningTime="2026-10-30 21:34:06.746109619 +0000 UTC m=+67.927229974"
Oct 30 21:34:08.645837 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:08.644788    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6n8r" event={"ID":"8953ad86-e744-4251-99cc-278bc81abe74","Type":"ContainerStarted","Data":"efb1dc6e6e6c80a6b99f9b2b442cf1c01a8a8969f5912de78fbdd4614710df26"}
Oct 30 21:34:08.645837 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:08.644882    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6n8r" event={"ID":"8953ad86-e744-4251-99cc-278bc81abe74","Type":"ContainerStarted","Data":"8314ce4affab5204d0dc75d005a861c50de37acfda32014be07b12da8a63896a"}
Oct 30 21:34:08.759967 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:08.759909    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t6n8r" podStartSLOduration=67.121763441 podStartE2EDuration="1m8.759887784s" totalImagesPullingTime="1.638124343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:08.758184453 +0000 UTC m=+69.939304807" watchObservedRunningTime="2026-10-30 21:34:08.759887784 +0000 UTC m=+69.941008139"
Oct 30 21:34:09.657597 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:09.657559    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf" event={"ID":"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7","Type":"ContainerStarted","Data":"82ddce8cdc8b58e42fe286b109028586b061033ce6e7c7eb8255f1d2d98265a7"}
Oct 30 21:34:09.658011 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:09.657659    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf" event={"ID":"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7","Type":"ContainerStarted","Data":"64f02f5e742acb1654bd1853e2cd9d268b9bfbb6f5565de532821feb9d285bf5"}
Oct 30 21:34:09.658011 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:09.657692    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf" event={"ID":"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7","Type":"ContainerStarted","Data":"d9a00bf39b4b8a7ac4e2f051feaa26f55b2334f3dc8048520d672d0459b7ef84"}
Oct 30 21:34:09.881884 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:09.881824    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf" podStartSLOduration=1.952958485 podStartE2EDuration="7.881808612s" totalImagesPullingTime="5.928850127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:09.83098927 +0000 UTC m=+71.012109679" watchObservedRunningTime="2026-10-30 21:34:09.881808612 +0000 UTC m=+71.062928966"
Oct 30 21:34:10.666163 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:10.666127    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6gtf" event={"ID":"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10","Type":"ContainerStarted","Data":"18b1e626965b56ce647fafb24cffb116c3ce4c58486d27ae6f9d81ceb44b6f79"}
Oct 30 21:34:10.666714 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:10.666227    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:10.778183 ip-10-0-135-39 kubenswrapper[2586]: I1030 21:34:10.778133    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b6gtf" podStartSLOduration=66.773581279 podStartE2EDuration="1m10.778118951s" totalImagesPullingTime="4.004537672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:10.777015428 +0000 UTC m=+71.958135783" watchObservedRunningTime="2026-10-30 21:34:10.778118951 +0000 UTC m=+71.959239339"
Oct 30 21:34:11.659796 ip-10-0-135-39 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:34:11.669794 ip-10-0-135-39 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:34:11.669958 ip-10-0-135-39 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:34:11.670007 ip-10-0-135-39 systemd[1]: kubelet.service: Consumed 3.858s CPU time.
Oct 30 21:34:11.679911 ip-10-0-135-39 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:34:11.752082 ip-10-0-135-39 kubenswrapper[5561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:34:11.752082 ip-10-0-135-39 kubenswrapper[5561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:34:11.752082 ip-10-0-135-39 kubenswrapper[5561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:34:11.752082 ip-10-0-135-39 kubenswrapper[5561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:34:11.754170 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754094    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.754170 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754171    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754175    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754178    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754182    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754185    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754188    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754191    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754194    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754197    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754199    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754202    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754204    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754211    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754214    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754217    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754219    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754224    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754228    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754232    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.754239 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754234    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754238    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754241    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754244    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754247    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754250    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754253    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754255    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754258    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754261    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754264    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754266    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754269    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754272    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754274    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754278    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754283    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754286    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754289    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754292    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.754829 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754295    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754298    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754300    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754303    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754305    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754308    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754311    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754314    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754317    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754320    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754322    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754325    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754328    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754331    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754334    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754336    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754339    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754343    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754345    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754360    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.755335 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754363    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754366    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754369    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754372    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754374    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754377    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754380    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754383    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754386    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754389    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754391    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754394    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754397    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754399    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754402    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754405    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754407    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754410    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754413    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.755838 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754416    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754419    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754422    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754425    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754427    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754430    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754433    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754436    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754439    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754442    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754445    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754447    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754450    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754453    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754455    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754458    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754462    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754464    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754467    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.756280 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754469    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754472    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754475    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754477    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754480    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754483    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754486    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.754488    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755334    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755344    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755363    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755367    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755370    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755373    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755377    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755379    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755382    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755385    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755387    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755390    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.756749 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755393    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755397    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755401    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755404    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755407    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755410    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755413    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755415    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755418    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755420    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755423    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755426    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755429    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755431    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755434    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755436    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755439    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755442    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755446    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.757222 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755449    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755452    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755455    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755459    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755461    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755464    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755467    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755469    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755472    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755475    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755478    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755481    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755483    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755486    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755489    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755492    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755494    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755497    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755500    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755502    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.757725 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755505    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755507    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755510    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755512    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755515    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755518    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755520    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755523    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755526    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755530    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755533    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755536    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755538    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755541    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755544    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755546    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755549    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755551    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755554    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.758218 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755556    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755559    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755566    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755569    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755572    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755574    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755577    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755580    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755583    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755585    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755588    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755591    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755593    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755596    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755598    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755601    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755604    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755607    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755609    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755612    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.758690 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755615    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755618    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755620    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755623    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755626    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755628    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755631    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755633    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755636    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755639    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755641    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755643    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755646    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755649    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755651    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.755658    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756175    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756181    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756187    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756190    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.759197 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756193    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756196    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756198    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756201    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756204    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756206    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756209    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756211    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756214    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756217    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756220    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756223    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756226    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756229    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756231    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756234    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756236    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756239    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756242    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.759706 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756244    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756247    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756250    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756252    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756255    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756257    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756260    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756263    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756266    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756268    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756271    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756275    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756277    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756280    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756283    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756285    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756288    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756291    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756294    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.760189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756297    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756299    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756302    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756304    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756307    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756310    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756312    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756315    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756317    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756320    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756322    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756325    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756328    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756332    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756336    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756339    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756342    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756345    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756360    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.760661 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756363    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756366    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756369    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756378    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756381    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756385    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756387    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756390    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756393    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756397    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756400    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756403    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756405    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756408    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756411    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756413    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756416    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756419    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756421    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756424    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.761109 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756426    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756429    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756431    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756434    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756439    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756442    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756444    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756447    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756449    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756452    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756454    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756457    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756459    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756462    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756464    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756467    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756470    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756473    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756475    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756479    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.761664 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756482    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756484    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756487    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756489    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.756492    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756583    5561 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756598    5561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756605    5561 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756609    5561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756613    5561 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756617    5561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756621    5561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756625    5561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756628    5561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756632    5561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756635    5561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756642    5561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756645    5561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756648    5561 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756651    5561 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756662    5561 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756666    5561 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756669    5561 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:34:11.762561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756672    5561 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756675    5561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756679    5561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756682    5561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756686    5561 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756689    5561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756693    5561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756697    5561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756700    5561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756703    5561 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756706    5561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756709    5561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756712    5561 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756715    5561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756719    5561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756722    5561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756725    5561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756728    5561 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756731    5561 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756734    5561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756739    5561 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756742    5561 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756745    5561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756748    5561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:34:11.763490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756751    5561 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756757    5561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756760    5561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756763    5561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756767    5561 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756770    5561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756773    5561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756776    5561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756779    5561 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756782    5561 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756784    5561 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756787    5561 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756792    5561 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756795    5561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756798    5561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756802    5561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756805    5561 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756808    5561 flags.go:64] FLAG: --help="false"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756811    5561 flags.go:64] FLAG: --hostname-override="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756814    5561 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756817    5561 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756820    5561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756824    5561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:34:11.764413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756827    5561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756830    5561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756833    5561 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756836    5561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756839    5561 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756842    5561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756845    5561 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756848    5561 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756851    5561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756854    5561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756863    5561 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756866    5561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756869    5561 flags.go:64] FLAG: --lock-file=""
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756872    5561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756875    5561 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756878    5561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756887    5561 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756890    5561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756894    5561 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756896    5561 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756899    5561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756903    5561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756906    5561 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756909    5561 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756913    5561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:34:11.765056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756917    5561 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756921    5561 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756924    5561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756927    5561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756930    5561 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756933    5561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756936    5561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756939    5561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756942    5561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756949    5561 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756952    5561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756956    5561 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756959    5561 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756962    5561 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756965    5561 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756968    5561 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756971    5561 flags.go:64] FLAG: --port="10250"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756977    5561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756980    5561 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-09d8040e22cd6e3f9"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756984    5561 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756987    5561 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756990    5561 flags.go:64] FLAG: --register-node="true"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756993    5561 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756996    5561 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.756999    5561 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757002    5561 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:34:11.765747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757005    5561 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757009    5561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757012    5561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757015    5561 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757018    5561 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757021    5561 flags.go:64] FLAG: --runonce="false"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757024    5561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757028    5561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757031    5561 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757034    5561 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757037    5561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757040    5561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757043    5561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757046    5561 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757049    5561 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757052    5561 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757054    5561 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757057    5561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757060    5561 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757063    5561 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757066    5561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757071    5561 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757074    5561 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757080    5561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757083    5561 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:34:11.766492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757086    5561 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757089    5561 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757092    5561 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757095    5561 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757098    5561 flags.go:64] FLAG: --v="2"
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757103    5561 flags.go:64] FLAG: --version="false"
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757107    5561 flags.go:64] FLAG: --vmodule=""
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757111    5561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757114    5561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757248    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757252    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757254    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757257    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757260    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757262    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757265    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757269    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757271    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757274    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757277    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757280    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757282    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.767134 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757285    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757287    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757290    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757293    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757295    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757298    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757300    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757303    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757308    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757311    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757313    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757316    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757318    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757321    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757323    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757326    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757328    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757331    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757333    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.767790 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757336    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757339    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757341    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757344    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757346    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757363    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757366    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757369    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757371    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757374    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757377    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757379    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757382    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757385    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757387    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757390    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757392    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757395    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757398    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.768572 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757400    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757403    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757407    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757410    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757413    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757416    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757418    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757421    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757423    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757426    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757429    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757431    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757434    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757437    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757440    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757442    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757445    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757448    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757451    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757453    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.769091 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757456    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757459    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757461    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757464    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757467    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757469    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757472    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757474    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757478    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757481    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757485    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757488    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757490    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757493    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757497    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757500    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757502    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757505    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757508    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757510    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.769635 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757513    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757515    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757518    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757521    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757523    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757526    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757528    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757531    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757534    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757536    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757539    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757542    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757545    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757547    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757551    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.770124 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.757578    5561 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757612    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757617    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757620    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757623    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757625    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757630    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757634    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757638    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757642    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757646    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757653    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757657    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757660    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757663    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757666    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757669    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757671    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757674    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757678    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.770634 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757682    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757687    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757691    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757695    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757699    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757703    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757707    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757711    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757714    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757719    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757723    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757727    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757736    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757741    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757744    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757747    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757750    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757753    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757755    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.771426 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757758    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757761    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757764    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757766    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757770    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757773    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757775    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757778    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757781    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757783    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757786    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757789    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757792    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757794    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757797    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757799    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757802    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757804    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757807    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757810    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.772132 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757812    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757815    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757817    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757820    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757823    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757827    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757830    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757833    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757836    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757839    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757842    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757844    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757847    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757850    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757852    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757855    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757858    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757861    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757863    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757866    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.772921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757869    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757873    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757877    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757882    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757887    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757891    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757896    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757900    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757903    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757906    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757910    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757914    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757918    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757922    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757926    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757930    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757935    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757941    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757944    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757947    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.773728 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757951    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757953    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757956    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757959    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757962    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757964    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757967    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.757972    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.761371    5561 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.761385    5561 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.761404    5561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.761409    5561 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761479    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761484    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761488    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761492    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761494    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761497    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761500    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.774337 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761503    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761506    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761508    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761511    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761514    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761517    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761520    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761523    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761525    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761528    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761531    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761533    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761537    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761539    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761542    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761545    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761548    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761551    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761555    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.774972 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761559    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761562    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761566    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761570    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761575    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761579    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761583    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761586    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761589    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761591    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761594    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761597    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761600    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761602    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761605    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761608    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761610    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761613    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761615    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761618    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.775509 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761620    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761623    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761626    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761629    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761631    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761634    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761637    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761640    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761643    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761645    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761648    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761651    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761653    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761656    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761658    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761661    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761663    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761666    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761669    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761671    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.776106 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761674    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761677    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761680    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761683    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761685    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761688    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761691    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761693    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761696    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761699    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761701    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761704    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761707    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761709    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761713    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761715    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761718    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761721    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761723    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761726    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.776907 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761729    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761732    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761735    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761737    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761740    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761742    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761745    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761748    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761750    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761753    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761755    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761758    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761760    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761763    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761765    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761768    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761771    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761775    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761779    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.777687 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.761783    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.761811    5561 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762451    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762578    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762585    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762590    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762595    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762601    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762605    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762610    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762614    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762618    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762628    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762659    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762664    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762668    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.778393 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762672    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762676    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762680    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762684    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762689    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762693    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762697    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762703    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762707    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762716    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762720    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762724    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762728    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762732    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762737    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762742    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762751    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762756    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762761    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.778779 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762765    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762769    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762774    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762779    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762784    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762788    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762793    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762797    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762801    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762810    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762815    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762842    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762848    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762855    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762860    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762864    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762869    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762873    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762877    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762881    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.779233 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762886    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762890    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762899    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762903    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762907    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762911    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762915    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762920    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762924    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762928    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762932    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762936    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762940    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762944    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762949    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762958    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762962    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762967    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762971    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.779724 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762975    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762979    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762983    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762987    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.762991    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763016    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763020    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763025    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763033    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763038    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763042    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763046    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763050    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763054    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763058    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763062    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763066    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763076    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763080    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.780171 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763083    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763086    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763092    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763094    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763097    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763100    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763103    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763106    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763108    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763111    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763114    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763117    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763119    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763122    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763128    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.763167    5561 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:34:11.780675 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763245    5561 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763250    5561 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763254    5561 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763262    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763266    5561 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763269    5561 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763272    5561 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763275    5561 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763277    5561 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763280    5561 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763283    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763288    5561 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763291    5561 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763294    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763297    5561 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763301    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763304    5561 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763307    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763311    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763316    5561 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:11.781063 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763320    5561 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763324    5561 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763329    5561 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763333    5561 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763338    5561 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763341    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763344    5561 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763366    5561 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763371    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763376    5561 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763380    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763385    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763388    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763392    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763394    5561 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763410    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763445    5561 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763453    5561 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763458    5561 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:11.781548 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763580    5561 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763599    5561 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763604    5561 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763608    5561 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763611    5561 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763613    5561 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763616    5561 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763619    5561 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763624    5561 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763630    5561 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763634    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763637    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763641    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763644    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763647    5561 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763649    5561 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763652    5561 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763655    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763658    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763661    5561 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:11.782274 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763664    5561 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763667    5561 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763670    5561 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763672    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763675    5561 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763678    5561 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763680    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763683    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763686    5561 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763689    5561 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763692    5561 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763697    5561 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763700    5561 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763704    5561 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763707    5561 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763710    5561 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763713    5561 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763715    5561 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763718    5561 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:11.783122 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763721    5561 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763723    5561 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763726    5561 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763728    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763731    5561 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763734    5561 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763737    5561 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763740    5561 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763743    5561 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763745    5561 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763748    5561 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763751    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763753    5561 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763760    5561 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763764    5561 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763768    5561 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763771    5561 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763773    5561 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763776    5561 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:11.783849 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763778    5561 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763781    5561 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763784    5561 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763787    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763790    5561 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763795    5561 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763798    5561 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763801    5561 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:11.763803    5561 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.763877    5561 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.765222    5561 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.765283    5561 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.765537    5561 server.go:1014] "Starting client certificate rotation"
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.766166    5561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.767404    5561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.767461    5561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:28:00 +0000 UTC" deadline="2028-06-23 21:27:46.68392412 +0000 UTC"
Oct 30 21:34:11.784529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.767498    5561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14447h53m34.916429313s"
Oct 30 21:34:11.784927 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.767775    5561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:34:11.784927 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.769039    5561 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:34:11.784927 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.773213    5561 log.go:25] "Validated CRI v1 image API"
Oct 30 21:34:11.784927 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.774088    5561 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:34:11.784927 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.778378    5561 fs.go:135] Filesystem UUIDs: map[185c5ad3-dcd2-4c95-be36-c4f6f9acdd62:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cd87be85-80f8-4b2e-a581-4273b804eb31:/dev/nvme0n1p4]
Oct 30 21:34:11.785095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.778402    5561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1675e3e8eeb657d4413af16c63aa25191ad5b93a36907b74a28f07571872b869/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1675e3e8eeb657d4413af16c63aa25191ad5b93a36907b74a28f07571872b869/userdata/shm major:0 minor:249 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/295769846d6fe93a27d4a57475bc82f9409cbe3c045d7638591bb2735c10a329/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/295769846d6fe93a27d4a57475bc82f9409cbe3c045d7638591bb2735c10a329/userdata/shm major:0 minor:363 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2fe3dbecc316b7419d6178e32b7e772cd2779564da2434ee3c876b3b4c7d2f85/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2fe3dbecc316b7419d6178e32b7e772cd2779564da2434ee3c876b3b4c7d2f85/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3ba54b3e04e6c6dbfa54a19250ff49288928c615fd4262ba9b844101e183ad4b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3ba54b3e04e6c6dbfa54a19250ff49288928c615fd4262ba9b844101e183ad4b/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/40ec291de57f9ebe37fc138551f9879af974ab785cde76cf5a9387015d84cc42/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/40ec291de57f9ebe37fc138551f9879af974ab785cde76cf5a9387015d84cc42/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f8b281f87588eff41948af47992acdd755673eed00d037b961ad8ab032dd662/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f8b281f87588eff41948af47992acdd755673eed00d037b961ad8ab032dd662/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc/userdata/shm major:0 minor:301 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7ea26db45ccc7993a35faed6e41c0e4c02da4e4978f3e677ef60520cab2b2c7d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7ea26db45ccc7993a35faed6e41c0e4c02da4e4978f3e677ef60520cab2b2c7d/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/802552c285f1449d87ce5dda5c178118d47afbd1278607c3a4f7b93b253bfb24/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/802552c285f1449d87ce5dda5c178118d47afbd1278607c3a4f7b93b253bfb24/userdata/shm major:0 minor:325 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8cd6c3896abdef6848cab2b2a64c300a6767576463f7112c45974d9f82e82e2d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8cd6c3896abdef6848cab2b2a64c300a6767576463f7112c45974d9f82e82e2d/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8d0c954fbe02bdfbb278e454964a537b1d13b54760b3cf81a7232c36c75bc181/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8d0c954fbe02bdfbb278e454964a537b1d13b54760b3cf81a7232c36c75bc181/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8fb6ffd95f21a764074825c274986126293daa335d28532cc4e0abb479e2be95/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8fb6ffd95f21a764074825c274986126293daa335d28532cc4e0abb479e2be95/userdata/shm major:0 minor:369 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/946c6ac6e2d51e3044aeb1071499ef276fd76f06d76c623accf0366c1a8f6356/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/946c6ac6e2d51e3044aeb1071499ef276fd76f06d76c623accf0366c1a8f6356/userdata/shm major:0 minor:364 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/98e9824226645370648f6836cc0079e64161483d6eca28a14bac0313481c773a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/98e9824226645370648f6836cc0079e64161483d6eca28a14bac0313481c773a/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a000f4d35b63ea8cd6cd3e6b8283f54bba3b2b7e1dba6ccb9f125364034e41c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a000f4d35b63ea8cd6cd3e6b8283f54bba3b2b7e1dba6ccb9f125364034e41c1/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a3e9d03a03e2ddb5f06770708776619d2e85fd210eb6a4aeade3e6e34e592c75/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a3e9d03a03e2ddb5f06770708776619d2e85fd210eb6a4aeade3e6e34e592c75/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/baa186044f9912f0174d6c85ef3310d50ee4ea462c2f4e75e2da0b85faaa982f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/baa186044f9912f0174d6c85ef3310d50ee4ea462c2f4e75e2da0b85faaa982f/userdata/shm major:0 minor:284 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c1fc6e24b8f47ca8e605eb9f4e9604d14428ec9d240f3341b93dc35ec4c0dc42/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c1fc6e24b8f47ca8e605eb9f4e9604d14428ec9d240f3341b93dc35ec4c0dc42/userdata/shm major:0 minor:212 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625/userdata/shm major:0 minor:382 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa32f5a732587d04e8e5c6db93443a15cc22aa31b475b81ce3501983c99cbca9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa32f5a732587d04e8e5c6db93443a15cc22aa31b475b81ce3501983c99cbca9/userdata/shm major:0 minor:316 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~projected/kube-api-access-h2z7p:{mountpoint:/var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~projected/kube-api-access-h2z7p major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19c8fb74-659e-4087-a035-3f26fa17b33c/volumes/kubernetes.io~projected/kube-api-access-clrkq:{mountpoint:/var/lib/kubelet/pods/19c8fb74-659e-4087-a035-3f26fa17b33c/volumes/kubernetes.io~projected/kube-api-access-clrkq major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a7bdebe-605a-432d-8cc7-91fc5417158d/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/1a7bdebe-605a-432d-8cc7-91fc5417158d/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23c665c7-85e7-4249-87e6-412b16d9e317/volumes/kubernetes.io~projected/kube-api-access-l648n:{mountpoint:/var/lib/kubelet/pods/23c665c7-85e7-4249-87e6-412b16d9e317/volumes/kubernetes.io~projected/kube-api-access-l648n major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~projected/kube-api-access-25d5t:{mountpoint:/var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~projected/kube-api-access-25d5t major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f7a665f-7b9e-415b-8d52-b355e803032d/volumes/kubernetes.io~projected/kube-api-access-lb2b2:{mountpoint:/var/lib/kubelet/pods/2f7a665f-7b9e-415b-8d52-b355e803032d/volumes/kubernetes.io~projected/kube-api-access-lb2b2 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f7a665f-7b9e-415b-8d52-b355e803032d/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/2f7a665f-7b9e-415b-8d52-b355e803032d/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49415b6f-832d-4e10-b7f9-e0b773574b30/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/49415b6f-832d-4e10-b7f9-e0b773574b30/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/574060d7-1710-4556-8f71-347eac712504/volumes/kubernetes.io~projected/kube-api-access-k7fgn:{mountpoint:/var/lib/kubelet/pods/574060d7-1710-4556-8f71-347eac712504/volumes/kubernetes.io~projected/kube-api-access-k7fgn major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/574060d7-1710-4556-8f71-347eac712504/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/574060d7-1710-4556-8f71-347eac712504/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f/volumes/kubernetes.io~projected/kube-api-access-kd5bx:{mountpoint:/var/lib/kubelet/pods/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f/volumes/kubernetes.io~projected/kube-api-access-kd5bx major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8953ad86-e744-4251-99cc-278bc81abe74/volumes/kubernetes.io~projected/kube-api-access-829q5:{mountpoint:/var/lib/kubelet/pods/8953ad86-e744-4251-99cc-278bc81abe74/volumes/kubernetes.io~projected/kube-api-access-829q5 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8953ad86-e744-4251-99cc-278bc81abe74/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/8953ad86-e744-4251-99cc-278bc81abe74/volumes/kubernetes.io~secret/metrics-certs major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/alertmanager/9 major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/config-reloader/6 major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~empty-dir/config-out major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~projected/kube-api-access-rsw5v:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~projected/kube-api-access-rsw5v major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~projected/tls-assets major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/config-volume major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/web-config major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95784d9c-a51b-4836-b072-b2c12568392e/volumes/kubernetes.io~projected/kube-api-access-r47w2:{mountpoint:/var/lib/kubelet/pods/95784d9c-a51b-4836-b072-b2c12568392e/volumes/kubernetes.io~projected/kube-api-access-r47w2 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95784d9c-a51b-4836-b072-b2c12568392e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/95784d9c-a51b-4836-b072-b2c12568392e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~projected/kube-api-access-87zqp:{mountpoint:/var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~projected/kube-api-access-87zqp major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15/volumes/kubernetes.io~projected/kube-api-access-9mb9t:{mountpoint:/var/lib/kubelet/pods/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15/volumes/kubernetes.io~projected/kube-api-access-9mb9t major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~projected/kube-api-access-g2rfm:{mountpoint:/var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~projected/kube-api-access-g2rfm major:0 minor:248 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:247 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~projected/kube-api-access-f5f9n:{mountpoint:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~projected/kube-api-access-f5f9n major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10/volumes/kubernetes.io~projected/kube-api-access-8q5kq:{mountpoint:/var/lib/kubelet/pods/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10/volumes/kubernetes.io~projected/kube-api-access-8q5kq major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volumes/kubernetes.io~projected/kube-api-access-8bczv:{mountpoint:/var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volumes/kubernetes.io~projected/kube-api-access-8bczv major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e6157882-1c00-4bb2-9790-25ba55f8ceb7/volumes/kubernetes.io~projected/kube-api-access-lx9nz:{mountpoint:/var/lib/kubelet/pods/e6157882-1c00-4bb2-9790-25ba55f8ceb7/volumes/kubernetes.io~projected/kube-api-access-lx9nz major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~projected/kube-api-access-pxxzj:{mountpoint:/var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~projected/kube-api-access-pxxzj major:0 minor:381 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:378 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1/volumes/kubernetes.io~projected/kube-api-access-ctzfj:{mountpoint:/var/lib/kubelet/pods/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1/volumes/kubernetes.io~projected/kube-api-access-ctzfj major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~projected/kube-api-access-w77mn:{mountpoint:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~projected/kube-api-access-w77mn major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:314 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/291a2e692eec7880147d534c647fdd60311db24d2a3f0b64d5e3496856d9cfc0/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/1e9d0574c6d78337511697a01f75992615936672d8e7c5f22559bbf29b99e4e8/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/116f6a440da56b1f96635f69df25d7e63e90c0eb4d5f6566a15a3900767075c6/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/ecbad85b52f6ba8603d173785c98acb0c49a2967947b132d4af75f28c513dceb/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/0d55307560a540b8ee9f63daedc4d3394755c3d40ddfef3ff29f4d1798be85ab/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/8c78cc2a8536b928a4f85e39896a754d2f539c67868895c6a4f377db221bc486/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/812d3319c1c00c7aae9c1514d9f16a2a42b935b75df66e001848a01f8fc0a257/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/a03555defe54699ae41039a9749746404a08e37a8bb503589c8ac505ced57033/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/5408cd8f1e1801b411a5aeef55584671b5ae5dd2e382495e0eb30b8c442fb24d/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/c78c652648f113d62b2b52c0b54fcaee0464f04352b0a1d4f8d274237437058f/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/54aa65846e5ac0390f86ef19f2b380714c3069a52af715b5fe56a407e9a548fb/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/6fc98e27f7a99d3450bb8b0c3fd92a54e9ed20fdab1900620fed2eac456acb99/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/9e152caec66374c14e620ddb2a8d9d116718cf30c05849672ec61f68e275a81d/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/96c04592d9a2062d129691d19f031b5da33070294415ecb9c4c0b48c3e461a46/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/6436d5272f48b135313116c9983961f8a98f74afc09c5185adeda1fa6ab29f0f/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/7bc9180d7a173f17ca0dad552557eb06e2484652e6a3c940fdeacb5e7f3f4683/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/d51614271ce66ebc29a1087f7cb2c6594f972463b8dbdc0ac8fbdb5649e9bf19/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/ce3bab9e8fca040300fcc005e80690fe49e114c69d9c6ae6873bc4499a8fc69e/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/e83c577693ac8566ef99ce48e1c2a4b80d1ee590504e0108a6e4e5ea364ad6f7/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/e75d10753cd43188a8a9ac7428f41fad66e886fb1b455fe5c57a2669f0ec9de5/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/7339ca4884d789b6fe3d1710fa958e081deb087e17b46cbe17b2ccd7e3c37d57/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-216:{mountpoint:/var/lib/containers/storage/overlay/8ba94e0bcb609973816f5866d39fd12d891137d3f84265c6b2d0d9ae7398b496/merged major:0 minor:216 fsType:overlay blockSize:0} overlay_0-218:{mountpoint:/var/lib/containers/storage/overlay/21df10aa5ae4d2ad6d2ff5edf32d0d23ffa5ecb74a96912959ee8e319cdca037/merged major:0 minor:218 fsType:overlay blockSize:0} overlay_0-220:{mountpoint:/var/lib/containers/storage/overlay/8c1b064632fbc8b2cf2e579ddea13b696ac21b64369878610d9dc3114397d8f7/merged major:0 minor:220 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/ea3a3501dd47ba203451e6068f8be20a0b2e69a1675800579405cf802f08f51a/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/e8fc163b123706d90dabeaab2006c783b7916cfa71147205acdbb99d1b577e9d/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/72dcd27aeb0c9e3205e490d0ade2ed2ac0040d73ba9aff2e9c58bbe57956353f/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/b60c8e599f8baa017697e3505eeb65c48b5c5a966d8ff7fb765a21137788da63/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/529542dc795f09ee9d71a2d213e43d30157b92bde7f6349d4ab16c238ad80dfd/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/3da801fbc59d53392005ad54583363645dc94f38457c0ff6d09c7f9ed63b17d1/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/ae499c3cece1e29097036131d96f99cf700049c67256a18c947fb8aca46acdcc/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/600dae1edc6b4f656b6d9ca3178e870d6c13d565c9e3890c31f552d73f25010c/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/62302c222a46f5f20f8c5e770472ab02f8bc152f71a8f4c25e3df2ae90606e90/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/10675eeef10695138fa1b96154ce9bbd64040755717ba370228576813e19c68e/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/0e9c7da8dcec8f28e6fad346b13debe822c645709d40e19e3f4be36ff0e59eb4/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/e25a0fe899b33d6c0f4d7fa8b324e2cf2875945daf8dc53104d34bf10a7e7fb5/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/46d62f895a326b67c037c32c96246e35c5c69a3bd835c19f6bb46d03492c2772/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/b226622a64826420cfe5d2aa2faf94e29744dd78e535bb95c73b77afe0985bab/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/17cea56807736cb51d9f63c43b42dcadae5ed9fd147c23b0c28ba1410b8eb58b/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/9d949ab76010f7535f75c1527c278444203f2716e36351f40785694bcc0ee9fb/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/39859b40d22d2650fb5568d2446b681fb4338414583f5bedb47d00fcbecb8312/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/dd8eef03af311403ff8cb8a10246aed8159063a7c99e92543fcd03e260aba32e/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/9986536e16b3a99da6f5aaa9475b6b53b79683bb300303c4c553986992c0c058/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-337:{mountpoint:/var/lib/containers/storage/overlay/b0f0d1b77efe9e258e769c19c46ffef77224e952811260ae7e1d930232c65b6b/merged major:0 minor:337 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/529d27b6eb6f9a0e9e47e42c6e71806941eee8ed790b5d83e461272e821b95f4/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-347:{mountpoint:/var/lib/containers/storage/overlay/f2a5b5f89f2ef7ed97d05d53e5a738ce467f9b772bb6b3280f1b136f04324c4f/merged major:0 minor:347 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/2502b52652e32d634dd48c13c9ff452b5852d1cc365e86c54558e8bec8ea2d04/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-367:{mountpoint:/var/lib/containers/storage/overlay/d90c65f1ce7b0de3d43a4a53ecd3ce97f753562f0a2848bf1ff8cde1e2c56ad6/merged major:0 minor:367 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/9744c60852c6f51fecbd31a2652a5f64513b09f19341ad19d6912ba55557ade5/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/f943b77a54f307c3cc91e90005f4a2a6a1dc3f5e877ebc6df76ed187e50b6c21/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-384:{mountpoint:/var/lib/containers/storage/overlay/d20339672b1cd52ff33576b557b491d9b00cc915463c730216748d44e991cacf/merged major:0 minor:384 fsType:overlay blockSize:0} overlay_0-386:{mountpoint:/var/lib/containers/storage/overlay/08a936c751a53516445326ef49c84d843341ea83c6afa3a76b5d9cd04cbfda83/merged major:0 minor:386 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/982fccbdc1cfb22c942901c5d624133a684f3307d6ebba79c045b496b5ab61a7/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-390:{mountpoint:/var/lib/containers/storage/overlay/ff16c3e899325b0027053929812960bf7214301810023418cf3f7ef3fe2a0aec/merged major:0 minor:390 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/3765f82de0c194eb894a7317fc35d405537ef0ccf9c3ece335745953d7294da5/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/e81247c318a779471421c909284d66f0f37cd285c27ee56781e877a690bf254d/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-407:{mountpoint:/var/lib/containers/storage/overlay/71fb96484959289fb27067203d8e6833937959440d9b825312d4d9baf68637fc/merged major:0 minor:407 fsType:overlay blockSize:0} overlay_0-413:{mountpoint:/var/lib/containers/storage/overlay/e7f5a76a285c64eea7e5696ff2a4eca2aa7c5a8b1317f2fd9c054b3fcabe2995/merged major:0 minor:413 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/8daeef1523593cc2e6a05511b3ee21cf991c7a25c734478f5a61d3a456ef51bf/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/a011cac260b021759ced7715bcad9d2880b5c3cda72e900b07741a08f5438ded/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/bb00db351bcaf9876b0f81e8fc35f13ca7585e27dc21a5a89fb860e3418d859d/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/5c430f93c1bbd93f4f0323d9879b51dc58849625cf5c1005275e741279f9a1f4/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/332d930c7a3baf6826feb79385a28a4719291ce660c455f2174ed50ceb8dbe5b/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/c7d13a502c61d6f9bddabcd1b093e518b1307df0ae995d06ba84d5e4871c887e/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/4f1b691dff6afd052c296cb4002337de414e6bca320e08bd5699d00b8af990a3/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/2a00701d300340d3591e64363b6ddb86ca6d784df6ddd3dacc8e33fdcebd3cd8/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/2bc01ea8841996251517e9eeaeec8642025c268a8d81fda6e25142e8b94e2eda/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/00723801ee1547ef9ebd9a5d69a59736c8d58076be3e345bac08a0178dc845ae/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/53dc19c650eb437d52784a0b9ad55092fcfd864f44a51be0a8d40065bc412d02/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/71fb94cad7216e6623f94eb9314a290165cb2479f6dfee6a43be1f6d909c02cd/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 21:34:11.789182 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.788790    5561 manager.go:217] Machine: {Timestamp:2026-10-30 21:34:11.788301786 +0000 UTC m=+0.065995247 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100922 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d29ec51aaa8d5fa4b2a46c9aa4052 SystemUUID:ec2d29ec-51aa-a8d5-fa4b-2a46c9aa4052 BootID:0974c4e7-01d1-4683-ad1f-fff81392643f Filesystems:[{Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/574060d7-1710-4556-8f71-347eac712504/volumes/kubernetes.io~projected/kube-api-access-k7fgn DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:243 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:282 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~projected/kube-api-access-25d5t DeviceMajor:0 DeviceMinor:283 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-407 DeviceMajor:0 DeviceMinor:407 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a000f4d35b63ea8cd6cd3e6b8283f54bba3b2b7e1dba6ccb9f125364034e41c1/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:312 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:323 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a3e9d03a03e2ddb5f06770708776619d2e85fd210eb6a4aeade3e6e34e592c75/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:293 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~projected/kube-api-access-rsw5v DeviceMajor:0 DeviceMinor:295 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:293 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:288 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/23c665c7-85e7-4249-87e6-412b16d9e317/volumes/kubernetes.io~projected/kube-api-access-l648n DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-216 DeviceMajor:0 DeviceMinor:216 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8cd6c3896abdef6848cab2b2a64c300a6767576463f7112c45974d9f82e82e2d/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:290 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10/volumes/kubernetes.io~projected/kube-api-access-8q5kq DeviceMajor:0 DeviceMinor:362 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-218 DeviceMajor:0 DeviceMinor:218 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa32f5a732587d04e8e5c6db93443a15cc22aa31b475b81ce3501983c99cbca9/userdata/shm DeviceMajor:0 DeviceMinor:316 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/574060d7-1710-4556-8f71-347eac712504/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~projected/kube-api-access-w77mn DeviceMajor:0 DeviceMinor:315 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:289 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c1fc6e24b8f47ca8e605eb9f4e9604d14428ec9d240f3341b93dc35ec4c0dc42/userdata/shm DeviceMajor:0 DeviceMinor:212 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/946c6ac6e2d51e3044aeb1071499ef276fd76f06d76c623accf0366c1a8f6356/userdata/shm DeviceMajor:0 DeviceMinor:364 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e6157882-1c00-4bb2-9790-25ba55f8ceb7/volumes/kubernetes.io~projected/kube-api-access-lx9nz DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2fe3dbecc316b7419d6178e32b7e772cd2779564da2434ee3c876b3b4c7d2f85/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:294 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/baa186044f9912f0174d6c85ef3310d50ee4ea462c2f4e75e2da0b85faaa982f/userdata/shm DeviceMajor:0 DeviceMinor:284 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:293 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-390 DeviceMajor:0 DeviceMinor:390 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~projected/kube-api-access-h2z7p DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-220 DeviceMajor:0 DeviceMinor:220 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625/userdata/shm DeviceMajor:0 DeviceMinor:382 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:291 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~projected/kube-api-access-f5f9n DeviceMajor:0 DeviceMinor:324 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3ba54b3e04e6c6dbfa54a19250ff49288928c615fd4262ba9b844101e183ad4b/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7ea26db45ccc7993a35faed6e41c0e4c02da4e4978f3e677ef60520cab2b2c7d/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-413 DeviceMajor:0 DeviceMinor:413 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2f7a665f-7b9e-415b-8d52-b355e803032d/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2f7a665f-7b9e-415b-8d52-b355e803032d/volumes/kubernetes.io~projected/kube-api-access-lb2b2 DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:313 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/95784d9c-a51b-4836-b072-b2c12568392e/volumes/kubernetes.io~projected/kube-api-access-r47w2 DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:247 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:292 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:296 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1675e3e8eeb657d4413af16c63aa25191ad5b93a36907b74a28f07571872b869/userdata/shm DeviceMajor:0 DeviceMinor:249 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-337 DeviceMajor:0 DeviceMinor:337 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8fb6ffd95f21a764074825c274986126293daa335d28532cc4e0abb479e2be95/userdata/shm DeviceMajor:0 DeviceMinor:369 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~projected/kube-api-access-pxxzj DeviceMajor:0 DeviceMinor:381 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8953ad86-e744-4251-99cc-278bc81abe74/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:361 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f/volumes/kubernetes.io~projected/kube-api-access-kd5bx DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8953ad86-e744-4251-99cc-278bc81abe74/volumes/kubernetes.io~projected/kube-api-access-829q5 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c7eabbd5-c926-4eaf-9629-17b0da3ec706/volumes/kubernetes.io~projected/kube-api-access-g2rfm DeviceMajor:0 DeviceMinor:248 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-386 DeviceMajor:0 DeviceMinor:386 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/98e9824226645370648f6836cc0079e64161483d6eca28a14bac0313481c773a/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f8b281f87588eff41948af47992acdd755673eed00d037b961ad8ab032dd662/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:378 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f6990798-b558-45b4-86d2-ab0917893407/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:314 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1a7bdebe-605a-432d-8cc7-91fc5417158d/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:357 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/95784d9c-a51b-4836-b072-b2c12568392e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2be22202-fcae-4c90-b4ff-0eb07a3c41e0/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:278 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/40ec291de57f9ebe37fc138551f9879af974ab785cde76cf5a9387015d84cc42/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc/userdata/shm DeviceMajor:0 DeviceMinor:301 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:321 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8d0c954fbe02bdfbb278e454964a537b1d13b54760b3cf81a7232c36c75bc181/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1/volumes/kubernetes.io~projected/kube-api-access-ctzfj DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/295769846d6fe93a27d4a57475bc82f9409cbe3c045d7638591bb2735c10a329/userdata/shm DeviceMajor:0 DeviceMinor:363 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19c8fb74-659e-4087-a035-3f26fa17b33c/volumes/kubernetes.io~projected/kube-api-access-clrkq DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-367 DeviceMajor:0 DeviceMinor:367 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:377 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:297 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e0ceab3b-942e-4950-bb5e-96f62e431365/volumes/kubernetes.io~projected/kube-api-access-8bczv DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:289 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:322 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-347 DeviceMajor:0 DeviceMinor:347 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15/volumes/kubernetes.io~projected/kube-api-access-9mb9t DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/802552c285f1449d87ce5dda5c178118d47afbd1278607c3a4f7b93b253bfb24/userdata/shm DeviceMajor:0 DeviceMinor:325 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-384 DeviceMajor:0 DeviceMinor:384 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:293 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/49415b6f-832d-4e10-b7f9-e0b773574b30/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b66918c1-6a0a-495b-bb2e-df4e61590fe4/volumes/kubernetes.io~projected/kube-api-access-87zqp DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1675e3e8eeb657d MacAddress:4e:18:3b:97:11:d8 Speed:10000 Mtu:8901} {Name:295769846d6fe93 MacAddress:9e:b6:bb:ee:f0:9c Speed:10000 Mtu:8901} {Name:5cea08161858bd5 MacAddress:7a:fa:a6:67:c8:1e Speed:10000 Mtu:8901} {Name:7ea26db45ccc799 MacAddress:5a:79:2d:0b:64:a7 Speed:10000 Mtu:8901} {Name:802552c285f1449 MacAddress:b2:75:c3:4f:99:fd Speed:10000 Mtu:8901} {Name:8fb6ffd95f21a76 MacAddress:76:6f:68:19:4d:e0 Speed:10000 Mtu:8901} {Name:946c6ac6e2d51e3 MacAddress:42:19:d6:ae:d8:f6 Speed:10000 Mtu:8901} {Name:a3e9d03a03e2ddb MacAddress:a2:81:87:b5:f2:64 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:44:83:f3:00:05 Speed:0 Mtu:9001} {Name:br-int MacAddress:b6:db:28:42:39:d4 Speed:0 Mtu:8901} {Name:c1fc6e24b8f47ca MacAddress:8e:b6:86:63:5e:b4 Speed:10000 Mtu:8901} {Name:ec90556b38e0cbf MacAddress:ea:1a:cc:85:04:60 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:44:83:f3:00:05 Speed:0 Mtu:9001} {Name:f20eb2776a9e632 MacAddress:06:2f:8d:17:d2:53 Speed:10000 Mtu:8901} {Name:fa32f5a732587d0 MacAddress:e2:1c:d3:8a:bb:59 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:8a:6f:88:65:8f:c1 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:1e:7f:51:a2:e2:5c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:34:11.789182 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789184    5561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:34:11.789503 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789229    5561 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:34:11.789536 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789506    5561 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:34:11.789649 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789524    5561 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-39.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:34:11.789690 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789651    5561 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:34:11.789690 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789660    5561 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:34:11.789690 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789670    5561 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:34:11.789767 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789691    5561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:34:11.789795 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789778    5561 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:34:11.789938 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.789927    5561 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:34:11.790057 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.790047    5561 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:34:11.790092 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.790086    5561 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:34:11.790119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.790101    5561 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:34:11.790119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.790112    5561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:34:11.790184 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.790121    5561 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:34:11.790184 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.790131    5561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:34:11.791022 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.791004    5561 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:34:11.791022 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.791021    5561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:34:11.791589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.791571    5561 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:34:11.794389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794376    5561 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:34:11.794389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794389    5561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:34:11.794492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794483    5561 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:34:11.794678 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794667    5561 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:34:11.794708 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794697    5561 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:34:11.794785 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794777    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:34:11.794815 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794792    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:34:11.794815 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794799    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:34:11.794815 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794807    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:34:11.794815 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794814    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794821    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794831    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794840    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794851    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794860    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794870    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794885    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794914    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:34:11.794952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.794920    5561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:34:11.798231 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.798213    5561 server.go:1263] "Started kubelet"
Oct 30 21:34:11.798314 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.798286    5561 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:34:11.798687 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.798395    5561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:34:11.798687 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.798485    5561 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:34:11.798687 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.798777    5561 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:34:11.798775 ip-10-0-135-39 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:34:11.799982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.799950    5561 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.805462    5561 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.805668    5561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.805891    5561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:28:02 +0000 UTC" deadline="2028-07-01 13:46:21.018894386 +0000 UTC"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.805911    5561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14632h12m9.212986306s"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.806063    5561 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.806072    5561 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.806157    5561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:11.807415    5561 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.807781    5561 factory.go:55] Registering systemd factory
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.807799    5561 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.808090    5561 factory.go:153] Registering CRI-O factory
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.808103    5561 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.808151    5561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.808176    5561 factory.go:103] Registering Raw factory
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.808192    5561 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.808548    5561 manager.go:319] Starting recovery of all containers
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809006    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95784d9c-a51b-4836-b072-b2c12568392e" volumeName="kubernetes.io/projected/95784d9c-a51b-4836-b072-b2c12568392e-kube-api-access-r47w2" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809060    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6990798-b558-45b4-86d2-ab0917893407" volumeName="kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809074    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f7a665f-7b9e-415b-8d52-b355e803032d" volumeName="kubernetes.io/empty-dir/2f7a665f-7b9e-415b-8d52-b355e803032d-tmp-dir" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809085    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809096    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" volumeName="kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809107    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10" volumeName="kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809120    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0ceab3b-942e-4950-bb5e-96f62e431365" volumeName="kubernetes.io/projected/e0ceab3b-942e-4950-bb5e-96f62e431365-kube-api-access-8bczv" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809130    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809143    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809166    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809179    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f" volumeName="kubernetes.io/projected/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kube-api-access-kd5bx" seLinuxMountContext=""
Oct 30 21:34:11.809193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809193    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95784d9c-a51b-4836-b072-b2c12568392e" volumeName="kubernetes.io/configmap/95784d9c-a51b-4836-b072-b2c12568392e-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809205    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-federate-client-tls" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809232    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1" volumeName="kubernetes.io/projected/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-kube-api-access-ctzfj" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809246    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23c665c7-85e7-4249-87e6-412b16d9e317" volumeName="kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809256    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809266    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0ceab3b-942e-4950-bb5e-96f62e431365" volumeName="kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809277    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6990798-b558-45b4-86d2-ab0917893407" volumeName="kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809289    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809300    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0ceab3b-942e-4950-bb5e-96f62e431365" volumeName="kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809311    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e6157882-1c00-4bb2-9790-25ba55f8ceb7" volumeName="kubernetes.io/projected/e6157882-1c00-4bb2-9790-25ba55f8ceb7-kube-api-access-lx9nz" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809322    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809332    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6990798-b558-45b4-86d2-ab0917893407" volumeName="kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809344    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6990798-b558-45b4-86d2-ab0917893407" volumeName="kubernetes.io/empty-dir/f6990798-b558-45b4-86d2-ab0917893407-audit-log" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809381    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6990798-b558-45b4-86d2-ab0917893407" volumeName="kubernetes.io/projected/f6990798-b558-45b4-86d2-ab0917893407-kube-api-access-w77mn" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809410    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19c8fb74-659e-4087-a035-3f26fa17b33c" volumeName="kubernetes.io/configmap/19c8fb74-659e-4087-a035-3f26fa17b33c-serviceca" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809422    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" volumeName="kubernetes.io/empty-dir/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-textfile" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809435    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/projected/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-kube-api-access-f5f9n" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809453    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809466    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6990798-b558-45b4-86d2-ab0917893407" volumeName="kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809477    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" volumeName="kubernetes.io/projected/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-kube-api-access-25d5t" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809490    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809501    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95784d9c-a51b-4836-b072-b2c12568392e" volumeName="kubernetes.io/empty-dir/95784d9c-a51b-4836-b072-b2c12568392e-data-volume" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809513    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7eabbd5-c926-4eaf-9629-17b0da3ec706" volumeName="kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809526    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7eabbd5-c926-4eaf-9629-17b0da3ec706" volumeName="kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809541    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809555    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19c8fb74-659e-4087-a035-3f26fa17b33c" volumeName="kubernetes.io/projected/19c8fb74-659e-4087-a035-3f26fa17b33c-kube-api-access-clrkq" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809568    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95784d9c-a51b-4836-b072-b2c12568392e" volumeName="kubernetes.io/secret/95784d9c-a51b-4836-b072-b2c12568392e-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809579    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809591    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809602    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1" volumeName="kubernetes.io/configmap/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809615    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49415b6f-832d-4e10-b7f9-e0b773574b30" volumeName="kubernetes.io/secret/49415b6f-832d-4e10-b7f9-e0b773574b30-agent-certs" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809627    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8953ad86-e744-4251-99cc-278bc81abe74" volumeName="kubernetes.io/projected/8953ad86-e744-4251-99cc-278bc81abe74-kube-api-access-829q5" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809639    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809656    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809670    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-client-tls" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809683    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e6157882-1c00-4bb2-9790-25ba55f8ceb7" volumeName="kubernetes.io/empty-dir/e6157882-1c00-4bb2-9790-25ba55f8ceb7-tmp-dir" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809697    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809709    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23c665c7-85e7-4249-87e6-412b16d9e317" volumeName="kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809721    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f7a665f-7b9e-415b-8d52-b355e803032d" volumeName="kubernetes.io/projected/2f7a665f-7b9e-415b-8d52-b355e803032d-kube-api-access-lb2b2" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809732    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f7a665f-7b9e-415b-8d52-b355e803032d" volumeName="kubernetes.io/secret/2f7a665f-7b9e-415b-8d52-b355e803032d-metrics-tls" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809745    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809756    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" volumeName="kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809769    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" volumeName="kubernetes.io/projected/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-kube-api-access-9mb9t" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809780    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809793    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809814    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809850    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809862    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23c665c7-85e7-4249-87e6-412b16d9e317" volumeName="kubernetes.io/projected/23c665c7-85e7-4249-87e6-412b16d9e317-kube-api-access-l648n" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809876    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809887    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809900    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b66918c1-6a0a-495b-bb2e-df4e61590fe4" volumeName="kubernetes.io/projected/b66918c1-6a0a-495b-bb2e-df4e61590fe4-kube-api-access-87zqp" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809911    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809923    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="574060d7-1710-4556-8f71-347eac712504" volumeName="kubernetes.io/secret/574060d7-1710-4556-8f71-347eac712504-cert" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809934    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809946    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809958    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" volumeName="kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809969    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7eabbd5-c926-4eaf-9629-17b0da3ec706" volumeName="kubernetes.io/projected/c7eabbd5-c926-4eaf-9629-17b0da3ec706-kube-api-access-g2rfm" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809981    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7" volumeName="kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.809992    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e93dfe0f-83db-4682-a489-67ca3f81ee3b" volumeName="kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810005    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810019    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a7bdebe-605a-432d-8cc7-91fc5417158d" volumeName="kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810031    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" volumeName="kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810043    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" volumeName="kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810055    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" volumeName="kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-tls" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810066    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="574060d7-1710-4556-8f71-347eac712504" volumeName="kubernetes.io/projected/574060d7-1710-4556-8f71-347eac712504-kube-api-access-k7fgn" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810080    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8953ad86-e744-4251-99cc-278bc81abe74" volumeName="kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810097    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b66918c1-6a0a-495b-bb2e-df4e61590fe4" volumeName="kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-tuned" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810109    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b66918c1-6a0a-495b-bb2e-df4e61590fe4" volumeName="kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-tmp" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810121    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7eabbd5-c926-4eaf-9629-17b0da3ec706" volumeName="kubernetes.io/configmap/c7eabbd5-c926-4eaf-9629-17b0da3ec706-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810132    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f7a665f-7b9e-415b-8d52-b355e803032d" volumeName="kubernetes.io/configmap/2f7a665f-7b9e-415b-8d52-b355e803032d-config-volume" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810145    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810156    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810169    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0ceab3b-942e-4950-bb5e-96f62e431365" volumeName="kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-env-overrides" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810182    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0ceab3b-942e-4950-bb5e-96f62e431365" volumeName="kubernetes.io/secret/e0ceab3b-942e-4950-bb5e-96f62e431365-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810195    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6990798-b558-45b4-86d2-ab0917893407" volumeName="kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-client-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810207    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810219    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="170fa4df-5ce6-4243-b30a-7b2c91a733ce" volumeName="kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810230    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" volumeName="kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810244    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49415b6f-832d-4e10-b7f9-e0b773574b30" volumeName="kubernetes.io/configmap/49415b6f-832d-4e10-b7f9-e0b773574b30-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810258    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810270    5561 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" volumeName="kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810282    5561 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:34:11.810505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.810289    5561 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:34:11.836193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.836157    5561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:34:11.837742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.837713    5561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:34:11.837742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.837740    5561 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:34:11.837883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.837761    5561 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:34:11.837883 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:11.837802    5561 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:34:11.845066 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.845041    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:34:11.845165 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.845144    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:34:11.847959 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.847936    5561 generic.go:356] "Generic (PLEG): container finished" podID="2be22202-fcae-4c90-b4ff-0eb07a3c41e0" containerID="4c4b82d5e61e6aaffa87d7188e58abd4120a320dc9d619ee812a1b7dede6d6a8" exitCode=0
Oct 30 21:34:11.854198 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.854177    5561 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="44ff3699a7e23e811446fd1dc3d9ffb5646b526ca02140e2729b937301a27963" exitCode=0
Oct 30 21:34:11.854198 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.854195    5561 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="d3874591606539dc333a77e64a74f133466c18e0c165c18cf2209d8dd6d4d3d4" exitCode=0
Oct 30 21:34:11.854383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.854207    5561 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="d9f0898e0fb066f0e8a7ba7ffee05f0ad758e61d853914d44ff15bc8f85f4589" exitCode=0
Oct 30 21:34:11.854383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.854215    5561 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="b8587a80e6905c668304c4ef2a36081b5dfc0341219ece0c1c1f2508b65c8cd3" exitCode=0
Oct 30 21:34:11.854383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.854220    5561 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="1390a7d37fe5cc41f320afaa8ecacb0a3017d6f24f9c96572b9f2e43e13c6d9f" exitCode=0
Oct 30 21:34:11.854383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.854225    5561 generic.go:356] "Generic (PLEG): container finished" podID="c5cb4be6-9a9a-4318-bdb3-883bcbcffc15" containerID="99d84d1eed2c865ba4a695e4b869c81d1b32252545f1cd9571829eb1783c3a7b" exitCode=0
Oct 30 21:34:11.878984 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.878955    5561 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c" exitCode=0
Oct 30 21:34:11.885034 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.885016    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:34:11.885398 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.885374    5561 generic.go:356] "Generic (PLEG): container finished" podID="e0ceab3b-942e-4950-bb5e-96f62e431365" containerID="967a8f20482dc3c186c80be22f12d77899f50731e11603f5dca2f853ae5c091f" exitCode=1
Oct 30 21:34:11.900193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.900171    5561 generic.go:356] "Generic (PLEG): container finished" podID="af790951ab88ecd279296ee4eb7ed434" containerID="95d6d71dec84e692eee52bb89519bd5568ed4b92f16da144ad5c47a2900e2b66" exitCode=0
Oct 30 21:34:11.907490 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:11.907468    5561 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:34:11.915776 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.915722    5561 manager.go:324] Recovery completed
Oct 30 21:34:11.935064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.935046    5561 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:34:11.936676 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.936653    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:34:11.936755 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.936693    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:34:11.936755 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.936709    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:34:11.938208 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:11.938189    5561 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:34:11.938843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938829    5561 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:34:11.938843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938842    5561 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:34:11.938922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938844    5561 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:34:11.938922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938860    5561 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:34:11.938980 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938968    5561 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:34:11.939049 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938978    5561 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:34:11.939049 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938993    5561 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:34:11.939049 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.938999    5561 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:34:11.939049 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939004    5561 policy_none.go:50] "Start"
Oct 30 21:34:11.939049 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939011    5561 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:34:11.939049 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939020    5561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:34:11.939192 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939146    5561 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:34:11.939192 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939154    5561 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:34:11.939192 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939159    5561 policy_none.go:44] "Start"
Oct 30 21:34:11.939705 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939690    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:34:11.939748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939722    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:34:11.939748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.939737    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:34:11.943844 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.943822    5561 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:34:11.947583 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.947562    5561 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:34:11.947652 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:11.947585    5561 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:34:11.947652 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.947594    5561 server.go:87] "Starting device plugin registration server"
Oct 30 21:34:11.947909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.947896    5561 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:34:11.947951 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.947911    5561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:34:11.948011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.947992    5561 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:34:11.948136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.948095    5561 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:34:11.948206 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.948142    5561 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:34:11.948206 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.948150    5561 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:34:11.948293 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.948205    5561 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:34:11.948126671Z","UUID":"fd89e0fa-8456-4c88-9b6c-f29f43e9b925","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:34:11.949735 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:11.949716    5561 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:34:11.949825 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:11.949764    5561 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-39.us-west-1.compute.internal\" not found"
Oct 30 21:34:11.949825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.949719    5561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:34:11.949825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:11.949794    5561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:34:12.003060 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.003042    5561 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:34:12.048183 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.048168    5561 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:34:12.048987 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.048972    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:34:12.049059 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.048998    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:34:12.049059 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.049010    5561 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-39.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:34:12.049059 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.049060    5561 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.055604 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.055580    5561 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:12.095905 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.095888    5561 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:12.104184 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.104161    5561 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:12.104293 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.104266    5561 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:12.138482 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.138446    5561 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"]
Oct 30 21:34:12.139026 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.138996    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"9a9ea5e58d330910c40e8b399ca003fd","Type":"ContainerStarted","Data":"0659f086608e12816c7c1d86609ae661b9d971bae1048a391a64d66d71d1909f"}
Oct 30 21:34:12.139078 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.139033    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"9a9ea5e58d330910c40e8b399ca003fd","Type":"ContainerStarted","Data":"fc4648c23fd0c0295e9183cbe05ddb2baa833aedac4a4e882a6659556d9f441e"}
Oct 30 21:34:12.139078 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.139047    5561 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.139143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.139092    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"af790951ab88ecd279296ee4eb7ed434","Type":"ContainerStarted","Data":"d668f3673fcae9931f244336c382658bcd0863456ea10d84697aa3771075d93f"}
Oct 30 21:34:12.139143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.139106    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"af790951ab88ecd279296ee4eb7ed434","Type":"ContainerDied","Data":"95d6d71dec84e692eee52bb89519bd5568ed4b92f16da144ad5c47a2900e2b66"}
Oct 30 21:34:12.139143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.139118    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal" event={"ID":"af790951ab88ecd279296ee4eb7ed434","Type":"ContainerStarted","Data":"81a477e3dc73ec8f091b4aa8f48c69f9cc85f10fb7419cfa7d2494ab5a592598"}
Oct 30 21:34:12.139240 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.139213    5561 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.164836 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.164821    5561 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.164904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.164868    5561 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.203171 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.203115    5561 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:12.203171 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:12.203159    5561 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.204411 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.204395    5561 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:12.204478 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:12.204428    5561 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.304871 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.304841    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.304871 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.304875    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.305024 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.304891    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a9ea5e58d330910c40e8b399ca003fd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"9a9ea5e58d330910c40e8b399ca003fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.405629 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.405594    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.405629 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.405640    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.405839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.405666    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a9ea5e58d330910c40e8b399ca003fd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"9a9ea5e58d330910c40e8b399ca003fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.405839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.405724    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a9ea5e58d330910c40e8b399ca003fd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"9a9ea5e58d330910c40e8b399ca003fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.405839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.405737    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.405839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.405768    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af790951ab88ecd279296ee4eb7ed434-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" (UID: \"af790951ab88ecd279296ee4eb7ed434\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.790765 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.790728    5561 apiserver.go:52] "Watching apiserver"
Oct 30 21:34:12.895452 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.895425    5561 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:34:12.896077 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.896046    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-bfb94f7fb-rc2vj","openshift-image-registry/node-ca-sswwd","openshift-monitoring/alertmanager-main-0","openshift-monitoring/metrics-server-66b4c7697-8m46d","kube-system/global-pull-secret-syncer-2glxt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal","openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf","openshift-multus/multus-additional-cni-plugins-s5wqs","openshift-cluster-node-tuning-operator/tuned-24q2x","openshift-multus/multus-svcjf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b","kube-system/konnectivity-agent-qkddr","openshift-console/console-86bc8c7745-jf6t2","openshift-insights/insights-runtime-extractor-q94hx","openshift-monitoring/node-exporter-zqjjz","openshift-ingress-canary/ingress-canary-pzsfj","openshift-multus/network-metrics-daemon-t6n8r","openshift-network-diagnostics/network-check-target-b6gtf","openshift-dns/node-resolver-zmw2g","kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal","openshift-dns/dns-default-tvrlz","openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb","openshift-network-operator/iptables-alerter-dgbt5","openshift-ovn-kubernetes/ovnkube-node-6hsgh"]
Oct 30 21:34:12.906845 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.906819    5561 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.907014 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.906856    5561 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:12.909211 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909189    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95784d9c-a51b-4836-b072-b2c12568392e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:12.909316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909216    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r47w2\" (UniqueName: \"kubernetes.io/projected/95784d9c-a51b-4836-b072-b2c12568392e-kube-api-access-r47w2\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:12.909316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909238    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-run\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.909316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909263    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.909316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909288    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.909316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909313    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-netns\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909338    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909384    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-multus-daemon-config\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909411    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-iptables-alerter-script\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909435    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-host-slash\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909457    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95784d9c-a51b-4836-b072-b2c12568392e-data-volume\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909552    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7fgn\" (UniqueName: \"kubernetes.io/projected/574060d7-1710-4556-8f71-347eac712504-kube-api-access-k7fgn\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909567    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-modprobe-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.909589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909582    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909592    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95784d9c-a51b-4836-b072-b2c12568392e-data-volume\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909598    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909632    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-kubernetes\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909645    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-8664b77bf7-dmlx7"]
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909659    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49415b6f-832d-4e10-b7f9-e0b773574b30-agent-certs\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909736    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-socket-dir-parent\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909754    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909807    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-ovn\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909861    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-node-log\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909890    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-config\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909915    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-env-overrides\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.909958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909940    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c8fb74-659e-4087-a035-3f26fa17b33c-host\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909968    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd5bx\" (UniqueName: \"kubernetes.io/projected/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kube-api-access-kd5bx\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.909996    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910020    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-k8s-cni-cncf-io\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910047    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-multus-certs\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910072    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-host\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910097    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-binary-copy\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910122    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2z7p\" (UniqueName: \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910145    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0ceab3b-942e-4950-bb5e-96f62e431365-ovn-node-metrics-cert\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910203    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-netns\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910315    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910365    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-etc-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910389    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910439    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-conf\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910454    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-sys\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910487    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-tuned\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910541    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-kubelet\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910588    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-hostroot\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.910616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910625    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-tuned\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910645    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/574060d7-1710-4556-8f71-347eac712504-cert\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910682    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7a665f-7b9e-415b-8d52-b355e803032d-metrics-tls\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910752    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910783    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-systemd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910809    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-var-lib-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910833    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-multus\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910857    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-socket-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910879    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-var-lib-kubelet\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910911    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910960    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mb9t\" (UniqueName: \"kubernetes.io/projected/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-kube-api-access-9mb9t\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.910998    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a665f-7b9e-415b-8d52-b355e803032d-config-volume\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911015    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-tmp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911038    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f7a665f-7b9e-415b-8d52-b355e803032d-tmp-dir\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911062    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb2b2\" (UniqueName: \"kubernetes.io/projected/2f7a665f-7b9e-415b-8d52-b355e803032d-kube-api-access-lb2b2\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911085    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bczv\" (UniqueName: \"kubernetes.io/projected/e0ceab3b-942e-4950-bb5e-96f62e431365-kube-api-access-8bczv\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911117    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b66918c1-6a0a-495b-bb2e-df4e61590fe4-tmp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911153    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f7a665f-7b9e-415b-8d52-b355e803032d-tmp-dir\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911164    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysconfig\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911192    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-bin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911209    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-kubelet\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911259    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95784d9c-a51b-4836-b072-b2c12568392e-crio-socket\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:12.911321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911301    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95784d9c-a51b-4836-b072-b2c12568392e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911341    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrkq\" (UniqueName: \"kubernetes.io/projected/19c8fb74-659e-4087-a035-3f26fa17b33c-kube-api-access-clrkq\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911453    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-kubelet-config\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911475    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6157882-1c00-4bb2-9790-25ba55f8ceb7-tmp-dir\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911492    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9nz\" (UniqueName: \"kubernetes.io/projected/e6157882-1c00-4bb2-9790-25ba55f8ceb7-kube-api-access-lx9nz\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911508    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-system-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911527    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-cni-binary-copy\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911551    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-conf-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911581    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6157882-1c00-4bb2-9790-25ba55f8ceb7-tmp-dir\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911652    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cnibin\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911695    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911733    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911756    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctzfj\" (UniqueName: \"kubernetes.io/projected/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-kube-api-access-ctzfj\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911781    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-dbus\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911806    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87zqp\" (UniqueName: \"kubernetes.io/projected/b66918c1-6a0a-495b-bb2e-df4e61590fe4-kube-api-access-87zqp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911826    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911841    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911858    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-log-socket\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911884    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911926    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6157882-1c00-4bb2-9790-25ba55f8ceb7-hosts-file\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911954    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-etc-kubernetes\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.911991    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l648n\" (UniqueName: \"kubernetes.io/projected/23c665c7-85e7-4249-87e6-412b16d9e317-kube-api-access-l648n\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912032    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19c8fb74-659e-4087-a035-3f26fa17b33c-serviceca\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912049    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-device-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912087    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-sys-fs\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912115    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912152    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-systemd\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912186    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49415b6f-832d-4e10-b7f9-e0b773574b30-konnectivity-ca\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912212    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912252    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.912254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912270    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912313    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912408    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-systemd-units\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912444    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-bin\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912472    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-netd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912514    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-registration-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912539    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-lib-modules\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912564    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-system-cni-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912593    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-829q5\" (UniqueName: \"kubernetes.io/projected/8953ad86-e744-4251-99cc-278bc81abe74-kube-api-access-829q5\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912633    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-slash\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912696    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-script-lib\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.913280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.912761    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-cnibin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.918088 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.918071    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:12.932721 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.932689    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8664b77bf7-dmlx7"]
Oct 30 21:34:12.950541 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.950519    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:34:12.950972 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.950950    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.951073 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951059    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:34:12.951131 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951058    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:34:12.951131 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951088    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.951254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951227    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.951254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951234    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:34:12.951254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951273    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.951254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951284    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.951254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951248    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.951254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951278    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.951852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951754    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.951944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951859    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.951944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951905    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:34:12.952062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.951970    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-cni-binary-copy\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.952315 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952270    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.952564 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952548    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:34:12.952635 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952622    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:34:12.952692 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952656    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.952756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952686    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2v7jg\""
Oct 30 21:34:12.952810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952773    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qlt7z\""
Oct 30 21:34:12.952810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952787    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pplxq\""
Oct 30 21:34:12.952905 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952863    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wn5fv\""
Oct 30 21:34:12.952905 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.952875    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19c8fb74-659e-4087-a035-3f26fa17b33c-serviceca\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:34:12.953014 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953000    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:34:12.953050 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953044    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:34:12.953079 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953045    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zs2mb\""
Oct 30 21:34:12.953079 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953067    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tvlkp\""
Oct 30 21:34:12.953131 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953087    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mzgn4\""
Oct 30 21:34:12.953189 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953175    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:34:12.953346 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953335    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-cprz9\""
Oct 30 21:34:12.953464 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.953453    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:34:12.960805 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.960782    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-env-overrides\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.960883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.960844    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-binary-copy\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.960883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.960867    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0ceab3b-942e-4950-bb5e-96f62e431365-ovn-node-metrics-cert\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.960883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.960873    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c665c7-85e7-4249-87e6-412b16d9e317-multus-daemon-config\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:12.961011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.960904    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49415b6f-832d-4e10-b7f9-e0b773574b30-agent-certs\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:34:12.961011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.960916    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a7bdebe-605a-432d-8cc7-91fc5417158d-original-pull-secret\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:12.961011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.960904    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-config\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.963662 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.963636    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49415b6f-832d-4e10-b7f9-e0b773574b30-konnectivity-ca\") pod \"konnectivity-agent-qkddr\" (UID: \"49415b6f-832d-4e10-b7f9-e0b773574b30\") " pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:34:12.963746 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.963679    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0ceab3b-942e-4950-bb5e-96f62e431365-ovnkube-script-lib\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:12.984749 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984723    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.984843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984760    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.984843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984813    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:34:12.984843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984819    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:34:12.984843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984834    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:34:12.984843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984819    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:34:12.985015 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984854    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.985015 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.984865    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:34:12.990212 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.990195    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-iptables-alerter-script\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:34:12.991299 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.991285    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:12.993498 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:12.993482    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.001214 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001166    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:34:13.001388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001362    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:34:13.001388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001373    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:34:13.001515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001415    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:34:13.001515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001472    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:34:13.001515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001483    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:34:13.001515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001503    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:34:13.001656 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001564    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:34:13.001901 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001889    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:34:13.001988 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001937    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:34:13.001988 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.001973    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:13.002055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.002026    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:34:13.002055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.002081    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:13.002176 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.002162    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:34:13.002265 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.002175    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:34:13.002265 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.002198    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:34:13.002423 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.002313    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:34:13.007838 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.007822    5561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:34:13.010378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.010340    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:13.012241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.012227    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f7a665f-7b9e-415b-8d52-b355e803032d-config-volume\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:13.013514 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013497    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.013571 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013521    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-dbus\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:13.013571 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013537    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-log-socket\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.013571 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013553    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6157882-1c00-4bb2-9790-25ba55f8ceb7-hosts-file\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:34:13.013571 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013567    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-etc-kubernetes\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013602    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-etc-kubernetes\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013626    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-log-socket\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013635    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-dbus\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013658    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-bin\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013661    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6157882-1c00-4bb2-9790-25ba55f8ceb7-hosts-file\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013684    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-netd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013693    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-bin\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013701    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-metrics-client-ca\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013718    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.013744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013735    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsw5v\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013751    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-system-cni-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013767    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-slash\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013779    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013813    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-system-cni-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013816    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-slash\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013819    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-cni-netd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013836    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013873    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-metrics-client-ca\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013882    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013915    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.013942    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014016    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014044    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-run\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014084    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-netns\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014096    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014112    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-host-slash\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014122    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-run\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014143    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-netns\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014160    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-host-slash\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014176    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-modprobe-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014198    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-node-log\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014218    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014234    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6990798-b558-45b4-86d2-ab0917893407-audit-log\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014251    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-service-ca\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014265    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-modprobe-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014268    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014298    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-node-log\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014304    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c8fb74-659e-4087-a035-3f26fa17b33c-host\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014319    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6990798-b558-45b4-86d2-ab0917893407-audit-log\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014334    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014345    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c8fb74-659e-4087-a035-3f26fa17b33c-host\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014378    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-console-config\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014399    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f9n\" (UniqueName: \"kubernetes.io/projected/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-kube-api-access-f5f9n\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014447    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014477    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-host\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014506    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014530    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-host\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014534    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-netns\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014542    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014561    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-etc-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014587    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-netns\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014597    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014624    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-sys\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014597    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-etc-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014650    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-conf\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014754    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-sys\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014763    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-conf\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014780    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-kubelet\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014799    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-sys\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014806    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-client-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014822    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-kubelet\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014834    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mj9b\" (UniqueName: \"kubernetes.io/projected/42452a26-0fab-4ba7-acfb-c6781234a18f-kube-api-access-7mj9b\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014869    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7eabbd5-c926-4eaf-9629-17b0da3ec706-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014897    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-var-lib-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.014885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014914    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014929    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-federate-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014949    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-var-lib-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014948    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-var-lib-kubelet\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014974    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77mn\" (UniqueName: \"kubernetes.io/projected/f6990798-b558-45b4-86d2-ab0917893407-kube-api-access-w77mn\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014988    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-var-lib-kubelet\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.014990    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42452a26-0fab-4ba7-acfb-c6781234a18f-tmp\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015055    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015070    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015062    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7eabbd5-c926-4eaf-9629-17b0da3ec706-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015109    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2rfm\" (UniqueName: \"kubernetes.io/projected/c7eabbd5-c926-4eaf-9629-17b0da3ec706-kube-api-access-g2rfm\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015171    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-oauth-config\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015208    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-root\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015254    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysconfig\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015289    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-bin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015312    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-kubelet\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015314    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-bin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015336    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-kubelet-config\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015361    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysconfig\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015379    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cnibin\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015391    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-kubelet\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015401    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-system-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015401    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a7bdebe-605a-432d-8cc7-91fc5417158d-kubelet-config\") pod \"global-pull-secret-syncer-2glxt\" (UID: \"1a7bdebe-605a-432d-8cc7-91fc5417158d\") " pod="kube-system/global-pull-secret-syncer-2glxt"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015414    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-cnibin\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015420    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015442    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015447    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-system-cni-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015458    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015466    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015474    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-serving-certs-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015492    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015497    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-openvswitch\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015511    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015528    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015539    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015551    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-device-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015581    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-sys-fs\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015609    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015617    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-sys-fs\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015585    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-device-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015639    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-systemd\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015649    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-serving-certs-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015658    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-systemd-units\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015692    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-systemd-units\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015690    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015725    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-registration-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015741    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015745    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-sysctl-d\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015693    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-systemd\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015768    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-registration-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015751    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-lib-modules\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015826    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-client-certs\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015841    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-lib-modules\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015856    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-tls\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015884    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015939    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxxzj\" (UniqueName: \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015958    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-cnibin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015974    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.015990    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016006    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-textfile\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016022    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-wtmp\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016060    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016091    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-cnibin\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016093    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25d5t\" (UniqueName: \"kubernetes.io/projected/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-kube-api-access-25d5t\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016108    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-textfile\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016122    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-ovn\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016157    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-kubernetes\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016185    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-socket-dir-parent\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016156    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-ovn\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016193    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b66918c1-6a0a-495b-bb2e-df4e61590fe4-etc-kubernetes\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016209    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016227    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016237    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-socket-dir-parent\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016245    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016332    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-k8s-cni-cncf-io\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016367    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-multus-certs\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016384    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-k8s-cni-cncf-io\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016393    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-metrics-server-audit-profiles\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016422    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.016361 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016427    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-run-multus-certs\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016449    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016453    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016476    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016503    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016503    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016526    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016568    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-tls\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016590    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-hostroot\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016605    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-systemd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016626    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-oauth-serving-cert\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016632    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016650    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-hostroot\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016656    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-multus\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016670    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-run-systemd\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016694    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-socket-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016705    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-host-var-lib-cni-multus\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016721    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-trusted-ca-bundle\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016747    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016763    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-socket-dir\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016778    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016833    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-metrics-client-ca\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016850    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95784d9c-a51b-4836-b072-b2c12568392e-crio-socket\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016870    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016887    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-conf-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016902    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016919    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-serving-cert\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016942    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c665c7-85e7-4249-87e6-412b16d9e317-multus-conf-dir\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016963    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95784d9c-a51b-4836-b072-b2c12568392e-crio-socket\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.016967    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0ceab3b-942e-4950-bb5e-96f62e431365-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.017021    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-metrics-client-ca\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.018577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.017419    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.052265 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052210    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:34:13.052411 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052393    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:34:13.052570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052423    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:34:13.052570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052430    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:34:13.052570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052459    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 21:34:13.052570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052492    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:34:13.052570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052536    5561 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:13.052835 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:13.052577    5561 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:13.052891 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.052581    5561 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:13.053032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053013    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9nz\" (UniqueName: \"kubernetes.io/projected/e6157882-1c00-4bb2-9790-25ba55f8ceb7-kube-api-access-lx9nz\") pod \"node-resolver-zmw2g\" (UID: \"e6157882-1c00-4bb2-9790-25ba55f8ceb7\") " pod="openshift-dns/node-resolver-zmw2g"
Oct 30 21:34:13.053113 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:13.053097    5561 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-39.us-west-1.compute.internal"
Oct 30 21:34:13.053172 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053112    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:34:13.053172 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053115    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l648n\" (UniqueName: \"kubernetes.io/projected/23c665c7-85e7-4249-87e6-412b16d9e317-kube-api-access-l648n\") pod \"multus-svcjf\" (UID: \"23c665c7-85e7-4249-87e6-412b16d9e317\") " pod="openshift-multus/multus-svcjf"
Oct 30 21:34:13.053172 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053157    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd5bx\" (UniqueName: \"kubernetes.io/projected/6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f-kube-api-access-kd5bx\") pod \"aws-ebs-csi-driver-node-w2p8b\" (UID: \"6df4aa44-fd47-49f0-9ded-df9f9ad8fb8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w2p8b"
Oct 30 21:34:13.053305 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053211    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:34:13.053305 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053277    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 21:34:13.053422 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053341    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:34:13.053470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053457    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctzfj\" (UniqueName: \"kubernetes.io/projected/f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1-kube-api-access-ctzfj\") pod \"iptables-alerter-dgbt5\" (UID: \"f43da839-7e5b-4fe8-aa90-06fa9ee3fbc1\") " pod="openshift-network-operator/iptables-alerter-dgbt5"
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053572    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7fgn\" (UniqueName: \"kubernetes.io/projected/574060d7-1710-4556-8f71-347eac712504-kube-api-access-k7fgn\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053608    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053637    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2z7p\" (UniqueName: \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053692    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053747    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053764    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053791    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053884    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb2b2\" (UniqueName: \"kubernetes.io/projected/2f7a665f-7b9e-415b-8d52-b355e803032d-kube-api-access-lb2b2\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:13.053947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.053905    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mb9t\" (UniqueName: \"kubernetes.io/projected/c5cb4be6-9a9a-4318-bdb3-883bcbcffc15-kube-api-access-9mb9t\") pod \"multus-additional-cni-plugins-s5wqs\" (UID: \"c5cb4be6-9a9a-4318-bdb3-883bcbcffc15\") " pod="openshift-multus/multus-additional-cni-plugins-s5wqs"
Oct 30 21:34:13.054448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054129    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87zqp\" (UniqueName: \"kubernetes.io/projected/b66918c1-6a0a-495b-bb2e-df4e61590fe4-kube-api-access-87zqp\") pod \"tuned-24q2x\" (UID: \"b66918c1-6a0a-495b-bb2e-df4e61590fe4\") " pod="openshift-cluster-node-tuning-operator/tuned-24q2x"
Oct 30 21:34:13.054448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054194    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26jx5\""
Oct 30 21:34:13.054448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054282    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.054448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054298    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k7slv\""
Oct 30 21:34:13.054448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054313    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r47w2\" (UniqueName: \"kubernetes.io/projected/95784d9c-a51b-4836-b072-b2c12568392e-kube-api-access-r47w2\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:13.054448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054328    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:34:13.054448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054368    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-alkj5vgb4qp18\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054423    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrkq\" (UniqueName: \"kubernetes.io/projected/19c8fb74-659e-4087-a035-3f26fa17b33c-kube-api-access-clrkq\") pod \"node-ca-sswwd\" (UID: \"19c8fb74-659e-4087-a035-3f26fa17b33c\") " pod="openshift-image-registry/node-ca-sswwd"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054463    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bczv\" (UniqueName: \"kubernetes.io/projected/e0ceab3b-942e-4950-bb5e-96f62e431365-kube-api-access-8bczv\") pod \"ovnkube-node-6hsgh\" (UID: \"e0ceab3b-942e-4950-bb5e-96f62e431365\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054475    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054603    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054694    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054729    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-829q5\" (UniqueName: \"kubernetes.io/projected/8953ad86-e744-4251-99cc-278bc81abe74-kube-api-access-829q5\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054786    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054804    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054826    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wv6gv\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054865    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054877    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054889    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q5kq\" (UniqueName: \"kubernetes.io/projected/d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10-kube-api-access-8q5kq\") pod \"network-check-target-b6gtf\" (UID: \"d9fa252f-d4f0-4d3b-b2e2-cd9f7cf5fc10\") " pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054897    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054865    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7hq5v\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054928    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054941    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054945    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054964    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76thg\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054947    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054943    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-flltm\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054984    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054865    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pczpt\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054984    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jnpcj\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.054974    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055030    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-f6ntr\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055033    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055038    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055051    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6t698\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055062    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055116    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5hnkf\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055115    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-client-ca-bundle\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055162    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2n6f\""
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055164    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-federate-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055247    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055251    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.055273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055248    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rjmnn\""
Oct 30 21:34:13.056267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055741    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.056267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055767    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-telemeter-client-tls\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.056267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.055782    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c7eabbd5-c926-4eaf-9629-17b0da3ec706-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.057055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.056966    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-tls\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.057055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.056984    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.057055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.056990    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-tls\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.057055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.057006    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.057055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.057013    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.057055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.057016    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.057055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.057034    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.057333 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.056955    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.057333 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.057183    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6990798-b558-45b4-86d2-ab0917893407-secret-metrics-server-client-certs\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.057333 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.057199    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.057333 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.057236    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6990798-b558-45b4-86d2-ab0917893407-metrics-server-audit-profiles\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.060393 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.060374    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8953ad86-e744-4251-99cc-278bc81abe74-metrics-certs\") pod \"network-metrics-daemon-t6n8r\" (UID: \"8953ad86-e744-4251-99cc-278bc81abe74\") " pod="openshift-multus/network-metrics-daemon-t6n8r"
Oct 30 21:34:13.060468 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.060376    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95784d9c-a51b-4836-b072-b2c12568392e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:13.060503 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.060472    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:13.061507 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.061490    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:13.061573 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.061558    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7a665f-7b9e-415b-8d52-b355e803032d-metrics-tls\") pod \"dns-default-tvrlz\" (UID: \"2f7a665f-7b9e-415b-8d52-b355e803032d\") " pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:13.061573 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.061562    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/574060d7-1710-4556-8f71-347eac712504-cert\") pod \"ingress-canary-pzsfj\" (UID: \"574060d7-1710-4556-8f71-347eac712504\") " pod="openshift-ingress-canary/ingress-canary-pzsfj"
Oct 30 21:34:13.061636 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.061573    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95784d9c-a51b-4836-b072-b2c12568392e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q94hx\" (UID: \"95784d9c-a51b-4836-b072-b2c12568392e\") " pod="openshift-insights/insights-runtime-extractor-q94hx"
Oct 30 21:34:13.062716 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.062701    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\") pod \"console-bfb94f7fb-rc2vj\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") " pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:13.063876 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.063862    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.064987 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.064968    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.065024 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.064998    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.069715 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.069698    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5f9n\" (UniqueName: \"kubernetes.io/projected/d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7-kube-api-access-f5f9n\") pod \"telemeter-client-5cd5b84b84-5qqtf\" (UID: \"d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7\") " pod="openshift-monitoring/telemeter-client-5cd5b84b84-5qqtf"
Oct 30 21:34:13.069837 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.069821    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsw5v\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\") pod \"alertmanager-main-0\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:13.070077 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.070062    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2rfm\" (UniqueName: \"kubernetes.io/projected/c7eabbd5-c926-4eaf-9629-17b0da3ec706-kube-api-access-g2rfm\") pod \"prometheus-operator-9cd89b74c-h5pcb\" (UID: \"c7eabbd5-c926-4eaf-9629-17b0da3ec706\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-h5pcb"
Oct 30 21:34:13.070375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.070341    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w77mn\" (UniqueName: \"kubernetes.io/projected/f6990798-b558-45b4-86d2-ab0917893407-kube-api-access-w77mn\") pod \"metrics-server-66b4c7697-8m46d\" (UID: \"f6990798-b558-45b4-86d2-ab0917893407\") " pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:13.071237 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.071218    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25d5t\" (UniqueName: \"kubernetes.io/projected/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-kube-api-access-25d5t\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.071272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.071231    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxxzj\" (UniqueName: \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\") pod \"console-86bc8c7745-jf6t2\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") " pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:13.117841 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.117812    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-service-ca\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.117841 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.117845    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-console-config\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.117998 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.117865    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-sys\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.117998 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.117887    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mj9b\" (UniqueName: \"kubernetes.io/projected/42452a26-0fab-4ba7-acfb-c6781234a18f-kube-api-access-7mj9b\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.117998 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.117945    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-sys\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.118154 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118015    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42452a26-0fab-4ba7-acfb-c6781234a18f-tmp\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118154 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118059    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-oauth-config\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118154 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118085    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-root\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.118154 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118143    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-wtmp\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.118309 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118193    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-root\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.118309 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118195    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-oauth-serving-cert\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118309 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118234    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-trusted-ca-bundle\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118309 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118267    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-serving-cert\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118506 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118303    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2be22202-fcae-4c90-b4ff-0eb07a3c41e0-node-exporter-wtmp\") pod \"node-exporter-zqjjz\" (UID: \"2be22202-fcae-4c90-b4ff-0eb07a3c41e0\") " pod="openshift-monitoring/node-exporter-zqjjz"
Oct 30 21:34:13.118506 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118425    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42452a26-0fab-4ba7-acfb-c6781234a18f-tmp\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118607 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118537    5561 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:34:13.118659 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118643    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-service-ca\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118765 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118742    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-console-config\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.118937 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.118918    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-oauth-serving-cert\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.119262 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.119245    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-trusted-ca-bundle\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.120668 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.120639    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-serving-cert\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.120668 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.120652    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-oauth-config\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.174475 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.174449    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mj9b\" (UniqueName: \"kubernetes.io/projected/42452a26-0fab-4ba7-acfb-c6781234a18f-kube-api-access-7mj9b\") pod \"console-8664b77bf7-dmlx7\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") " pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.233144 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.233114    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:13.597227 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.597194    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8664b77bf7-dmlx7"]
Oct 30 21:34:13.600187 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:13.600157    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42452a26_0fab_4ba7_acfb_c6781234a18f.slice/crio-2938f7d0de7013c46230ffe734fba30d12b371043308b2449ee7776dcdc9e785 WatchSource:0}: Error finding container 2938f7d0de7013c46230ffe734fba30d12b371043308b2449ee7776dcdc9e785: Status 404 returned error can't find the container with id 2938f7d0de7013c46230ffe734fba30d12b371043308b2449ee7776dcdc9e785
Oct 30 21:34:13.911204 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.911165    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664b77bf7-dmlx7" event={"ID":"42452a26-0fab-4ba7-acfb-c6781234a18f","Type":"ContainerStarted","Data":"93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978"}
Oct 30 21:34:13.911613 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:13.911221    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664b77bf7-dmlx7" event={"ID":"42452a26-0fab-4ba7-acfb-c6781234a18f","Type":"ContainerStarted","Data":"2938f7d0de7013c46230ffe734fba30d12b371043308b2449ee7776dcdc9e785"}
Oct 30 21:34:14.025223 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:14.025184    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:14.428606 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:14.428578    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tvrlz"
Oct 30 21:34:14.843673 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:14.843563    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-8664b77bf7-dmlx7" podStartSLOduration=2.843549135 podStartE2EDuration="2.843549135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:14.843272767 +0000 UTC m=+3.120966243" watchObservedRunningTime="2026-10-30 21:34:14.843549135 +0000 UTC m=+3.121242672"
Oct 30 21:34:15.012017 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:15.011987    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:15.622254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:15.622215    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:15.626827 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:15.626803    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:15.921543 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:15.921474    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:34:16.063838 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:16.063810    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bfb94f7fb-rc2vj"]
Oct 30 21:34:16.421219 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:16.421195    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:18.874509 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:18.874484    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:18.874864 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:18.874689    5561 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:34:19.182261 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:19.182186    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:22.803314 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:22.803243    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:22.979281 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:22.979247    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:34:27.889721 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:27.889686    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c8d4d4b84-d8b7c"]
Oct 30 21:34:27.894066 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:27.894046    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:27.997890 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:27.997852    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8d4d4b84-d8b7c"]
Oct 30 21:34:28.027213 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027185    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-service-ca\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.027213 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027215    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-oauth-serving-cert\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.027420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027239    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-oauth-config\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.027420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027290    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-trusted-ca-bundle\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.027420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027337    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzqp5\" (UniqueName: \"kubernetes.io/projected/69a06e68-3e9e-4550-85b9-f3915a22f8b4-kube-api-access-tzqp5\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.027528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027423    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-config\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.027528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027459    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-serving-cert\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.027528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.027476    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69a06e68-3e9e-4550-85b9-f3915a22f8b4-tmp\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128600 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128572    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzqp5\" (UniqueName: \"kubernetes.io/projected/69a06e68-3e9e-4550-85b9-f3915a22f8b4-kube-api-access-tzqp5\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128712 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128612    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-config\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128712 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128636    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-serving-cert\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128712 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128652    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69a06e68-3e9e-4550-85b9-f3915a22f8b4-tmp\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128879 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128805    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-service-ca\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128879 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128850    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-oauth-serving-cert\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128908    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-oauth-config\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.128982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.128949    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-trusted-ca-bundle\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.129088 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.129067    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69a06e68-3e9e-4550-85b9-f3915a22f8b4-tmp\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.129589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.129560    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-config\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.129589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.129577    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-service-ca\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.129723 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.129617    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-oauth-serving-cert\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.130013 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.129990    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69a06e68-3e9e-4550-85b9-f3915a22f8b4-trusted-ca-bundle\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.131399 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.131377    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-serving-cert\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.131487 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.131468    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a06e68-3e9e-4550-85b9-f3915a22f8b4-console-oauth-config\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.185426 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.185377    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzqp5\" (UniqueName: \"kubernetes.io/projected/69a06e68-3e9e-4550-85b9-f3915a22f8b4-kube-api-access-tzqp5\") pod \"console-5c8d4d4b84-d8b7c\" (UID: \"69a06e68-3e9e-4550-85b9-f3915a22f8b4\") " pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.203828 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.203808    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:28.570384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.570301    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8d4d4b84-d8b7c"]
Oct 30 21:34:28.573305 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:28.573277    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a06e68_3e9e_4550_85b9_f3915a22f8b4.slice/crio-ae154eaa88262673c1157692395cf4cdfeed880e2d82e4ed2728acd44b392437 WatchSource:0}: Error finding container ae154eaa88262673c1157692395cf4cdfeed880e2d82e4ed2728acd44b392437: Status 404 returned error can't find the container with id ae154eaa88262673c1157692395cf4cdfeed880e2d82e4ed2728acd44b392437
Oct 30 21:34:28.963885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.963849    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8d4d4b84-d8b7c" event={"ID":"69a06e68-3e9e-4550-85b9-f3915a22f8b4","Type":"ContainerStarted","Data":"68d37b42bd955277a2aab3e1e25f0a8516d212fe59f31030394119b52c336dd3"}
Oct 30 21:34:28.964300 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:28.963902    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8d4d4b84-d8b7c" event={"ID":"69a06e68-3e9e-4550-85b9-f3915a22f8b4","Type":"ContainerStarted","Data":"ae154eaa88262673c1157692395cf4cdfeed880e2d82e4ed2728acd44b392437"}
Oct 30 21:34:29.080432 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:29.080380    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5c8d4d4b84-d8b7c" podStartSLOduration=2.080342055 podStartE2EDuration="2.080342055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:29.079898797 +0000 UTC m=+17.357592268" watchObservedRunningTime="2026-10-30 21:34:29.080342055 +0000 UTC m=+17.358035522"
Oct 30 21:34:31.644982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:31.644952    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hsgh"
Oct 30 21:34:32.016722 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:32.016653    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:33.266680 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.266646    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:34:33.267177 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.267034    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="alertmanager" containerID="cri-o://eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91" gracePeriod=120
Oct 30 21:34:33.267177 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.267072    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="prom-label-proxy" containerID="cri-o://1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2" gracePeriod=120
Oct 30 21:34:33.267177 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.267097    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="config-reloader" containerID="cri-o://50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626" gracePeriod=120
Oct 30 21:34:33.267177 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.267097    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-web" containerID="cri-o://bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e" gracePeriod=120
Oct 30 21:34:33.267177 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.267065    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy" containerID="cri-o://56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e" gracePeriod=120
Oct 30 21:34:33.267431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.267090    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-metric" containerID="cri-o://c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b" gracePeriod=120
Oct 30 21:34:33.983458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983417    5561 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2" exitCode=0
Oct 30 21:34:33.983458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983445    5561 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b" exitCode=0
Oct 30 21:34:33.983458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983454    5561 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e" exitCode=0
Oct 30 21:34:33.983458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983462    5561 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626" exitCode=0
Oct 30 21:34:33.983725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983470    5561 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91" exitCode=0
Oct 30 21:34:33.983725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983483    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2"}
Oct 30 21:34:33.983725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983546    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b"}
Oct 30 21:34:33.983725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983574    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e"}
Oct 30 21:34:33.983725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983597    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626"}
Oct 30 21:34:33.983725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:33.983620    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91"}
Oct 30 21:34:34.502622 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.502597    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:34.579496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579443    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579481    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579648 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579529    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579648 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579571    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579648 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579603    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579648 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579632    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579671    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579710    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579738    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579767    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.579852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579841    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.580111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579883    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.580111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579909    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:34.580111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579923    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\") pod \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\" (UID: \"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94\") "
Oct 30 21:34:34.580111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.579956    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:34.580306 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.580261    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-metrics-client-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.580306 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.580285    5561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-main-db\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.580453 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.580429    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:34.582896 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.582851    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:34.583030 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.582984    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:34.583128 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.583043    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "kube-api-access-rsw5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:34.583195 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.583137    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:34.583250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.583190    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:34.583311 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.583290    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:34.583527 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.583503    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:34.583815 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.583783    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:34.587711 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.587682    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:34.593816 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.593793    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config" pod "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" (UID: "8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:34.681470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681446    5561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681471    5561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681484    5561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681494    5561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681503    5561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681511    5561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-cluster-tls-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681537    5561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-web-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681546    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsw5v\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-kube-api-access-rsw5v\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681555    5561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-secret-alertmanager-main-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681565    5561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-config-out\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.681616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.681574    5561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94-tls-assets\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:34.990225 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.990188    5561 generic.go:356] "Generic (PLEG): container finished" podID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerID="bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e" exitCode=0
Oct 30 21:34:34.990368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.990258    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e"}
Oct 30 21:34:34.990368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.990290    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:34.990368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.990334    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94","Type":"ContainerDied","Data":"5cea08161858bd574d4acac407896ad4e2f1b55d4bae13f56a28439d8fe99ecc"}
Oct 30 21:34:34.990532 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.990381    5561 scope.go:122] "RemoveContainer" containerID="1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2"
Oct 30 21:34:34.998440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:34.998422    5561 scope.go:122] "RemoveContainer" containerID="c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b"
Oct 30 21:34:35.004983 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.004967    5561 scope.go:122] "RemoveContainer" containerID="56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e"
Oct 30 21:34:35.011269 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.011252    5561 scope.go:122] "RemoveContainer" containerID="bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e"
Oct 30 21:34:35.017610 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.017593    5561 scope.go:122] "RemoveContainer" containerID="50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626"
Oct 30 21:34:35.023814 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.023797    5561 scope.go:122] "RemoveContainer" containerID="eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91"
Oct 30 21:34:35.030063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.030046    5561 scope.go:122] "RemoveContainer" containerID="132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c"
Oct 30 21:34:35.036225 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.036209    5561 scope.go:122] "RemoveContainer" containerID="1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2"
Oct 30 21:34:35.036508 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:35.036488    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2\": container with ID starting with 1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2 not found: ID does not exist" containerID="1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2"
Oct 30 21:34:35.036567 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.036521    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2"} err="failed to get container status \"1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2\": rpc error: code = NotFound desc = could not find container \"1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2\": container with ID starting with 1f3340b735a00d6dbef1c6db82d44551ca75a093994005ccc3b46d03bea93ee2 not found: ID does not exist"
Oct 30 21:34:35.036567 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.036554    5561 scope.go:122] "RemoveContainer" containerID="c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b"
Oct 30 21:34:35.036792 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:35.036774    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b\": container with ID starting with c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b not found: ID does not exist" containerID="c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b"
Oct 30 21:34:35.036835 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.036800    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b"} err="failed to get container status \"c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b\": rpc error: code = NotFound desc = could not find container \"c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b\": container with ID starting with c7711b02785531c2bbbb8db6ff7b165a3428a421f187a4ff92cfef85836b6d6b not found: ID does not exist"
Oct 30 21:34:35.036835 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.036816    5561 scope.go:122] "RemoveContainer" containerID="56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e"
Oct 30 21:34:35.037044 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:35.037024    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e\": container with ID starting with 56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e not found: ID does not exist" containerID="56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e"
Oct 30 21:34:35.037107 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037055    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e"} err="failed to get container status \"56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e\": rpc error: code = NotFound desc = could not find container \"56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e\": container with ID starting with 56571f2e2d09d7b4b2311cba7e4893816b1745b98dc70f18f51087e39d78fb1e not found: ID does not exist"
Oct 30 21:34:35.037107 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037075    5561 scope.go:122] "RemoveContainer" containerID="bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e"
Oct 30 21:34:35.037293 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:35.037274    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e\": container with ID starting with bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e not found: ID does not exist" containerID="bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e"
Oct 30 21:34:35.037334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037298    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e"} err="failed to get container status \"bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e\": rpc error: code = NotFound desc = could not find container \"bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e\": container with ID starting with bb0c47b230ad40a4571bd52f296f9a53c2e4d12d7b9ca841dd1877936be8f44e not found: ID does not exist"
Oct 30 21:34:35.037334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037310    5561 scope.go:122] "RemoveContainer" containerID="50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626"
Oct 30 21:34:35.037537 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:35.037522    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626\": container with ID starting with 50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626 not found: ID does not exist" containerID="50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626"
Oct 30 21:34:35.037577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037544    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626"} err="failed to get container status \"50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626\": rpc error: code = NotFound desc = could not find container \"50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626\": container with ID starting with 50b10ae87483d72246d57016834e1bdfadd731a38cb601aa9adb22bb84514626 not found: ID does not exist"
Oct 30 21:34:35.037577 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037556    5561 scope.go:122] "RemoveContainer" containerID="eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91"
Oct 30 21:34:35.037728 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:35.037711    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91\": container with ID starting with eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91 not found: ID does not exist" containerID="eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91"
Oct 30 21:34:35.037768 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037734    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91"} err="failed to get container status \"eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91\": rpc error: code = NotFound desc = could not find container \"eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91\": container with ID starting with eb54aa23558f439da5ab530670bfb087689e25b1e5dc93b2ef5447f8be61af91 not found: ID does not exist"
Oct 30 21:34:35.037768 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037748    5561 scope.go:122] "RemoveContainer" containerID="132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c"
Oct 30 21:34:35.037922 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:35.037904    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c\": container with ID starting with 132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c not found: ID does not exist" containerID="132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c"
Oct 30 21:34:35.037956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.037929    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c"} err="failed to get container status \"132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c\": rpc error: code = NotFound desc = could not find container \"132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c\": container with ID starting with 132468172c5949bbbbed326c62b57fdc5c48e63b09972c60e3425a8b338d379c not found: ID does not exist"
Oct 30 21:34:35.157033 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.157009    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:34:35.207526 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.207504    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:34:35.207903 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.207885    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:34:35.208121 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208109    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="prom-label-proxy"
Oct 30 21:34:35.208159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208122    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="prom-label-proxy"
Oct 30 21:34:35.208159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208137    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="config-reloader"
Oct 30 21:34:35.208159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208142    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="config-reloader"
Oct 30 21:34:35.208159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208151    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-web"
Oct 30 21:34:35.208159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208155    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-web"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208165    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208170    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208176    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="alertmanager"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208181    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="alertmanager"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208192    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-metric"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208197    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-metric"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208206    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="init-config-reloader"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208210    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="init-config-reloader"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208249    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208256    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="alertmanager"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208262    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-metric"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208268    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="init-config-reloader"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208274    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="prom-label-proxy"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208281    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="config-reloader"
Oct 30 21:34:35.208325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.208286    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" containerName="kube-rbac-proxy-web"
Oct 30 21:34:35.213422 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.213406    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.268730 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.268661    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:34:35.268730 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.268672    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:34:35.268730 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.268700    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:34:35.268897 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.268788    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:34:35.268897 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.268791    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:34:35.268897 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.268843    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7hq5v\""
Oct 30 21:34:35.269267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.269251    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:34:35.269267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.269261    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:34:35.269562 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.269548    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:34:35.285726 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.285705    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-web-config\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.285779 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.285730    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.285779 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.285763    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.285861 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.285807    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.285912 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.285870    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.285961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.285934    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.286014 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.285970    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvsj4\" (UniqueName: \"kubernetes.io/projected/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-kube-api-access-qvsj4\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.286068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.286047    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.286116 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.286080    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.286186 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.286123    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.286186 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.286149    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-config-out\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.286260 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.286207    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-config-volume\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.286260 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.286226    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.319331 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.319306    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:34:35.323505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.323487    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:34:35.387008 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.386981    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387133 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387021    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387133 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387038    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387261 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387144    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvsj4\" (UniqueName: \"kubernetes.io/projected/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-kube-api-access-qvsj4\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387261 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387199    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387261 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387231    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387261 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387256    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387284    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-config-out\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387321    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-config-volume\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387343    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387400    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-web-config\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387423    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.387731 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387493    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.388011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.387981    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.388313 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.388178    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.388860 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.388838    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.390073 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.390050    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.390277 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.390255    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.390375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.390285    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.390687 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.390665    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-config-out\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.390860 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.390838    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.390932 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.390901    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-config-volume\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.390932 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.390925    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.392032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.392011    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.392032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.392028    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-web-config\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.444428 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.444401    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvsj4\" (UniqueName: \"kubernetes.io/projected/8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96-kube-api-access-qvsj4\") pod \"alertmanager-main-0\" (UID: \"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.534252 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.534204    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:34:35.654483 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.654458    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qkddr"
Oct 30 21:34:35.843810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:35.843740    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94" path="/var/lib/kubelet/pods/8b6b80f7-f5f7-4a39-adb2-2e4ff8e02d94/volumes"
Oct 30 21:34:36.034770 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:36.034741    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:34:36.037179 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:34:36.037152    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc7ae79_229c_4f3f_a83e_3b40b6d2ba96.slice/crio-2ca51d26b96903efda8ff01e0a58aa414728f45b1eddcfc62fd5d6047fbf4251 WatchSource:0}: Error finding container 2ca51d26b96903efda8ff01e0a58aa414728f45b1eddcfc62fd5d6047fbf4251: Status 404 returned error can't find the container with id 2ca51d26b96903efda8ff01e0a58aa414728f45b1eddcfc62fd5d6047fbf4251
Oct 30 21:34:36.999893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:36.999858    5561 generic.go:356] "Generic (PLEG): container finished" podID="8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96" containerID="7674e52f80d2cafe2d65f8f08fb1fb64cfcdb28e3577418ce9f093afd66b738f" exitCode=0
Oct 30 21:34:37.000275 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:36.999945    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerDied","Data":"7674e52f80d2cafe2d65f8f08fb1fb64cfcdb28e3577418ce9f093afd66b738f"}
Oct 30 21:34:37.000275 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:37.000025    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerStarted","Data":"2ca51d26b96903efda8ff01e0a58aa414728f45b1eddcfc62fd5d6047fbf4251"}
Oct 30 21:34:37.932700 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:37.932651    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-86bc8c7745-jf6t2" podUID="e93dfe0f-83db-4682-a489-67ca3f81ee3b" containerName="console" containerID="cri-o://ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786" gracePeriod=15
Oct 30 21:34:38.010069 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.010039    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerStarted","Data":"c4eb6fae32975545d8556a97583211db5a8fa7edb0b6d62920300f2415de8b29"}
Oct 30 21:34:38.010517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.010100    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerStarted","Data":"0dc8db75299f2455a947ef42c7381a940283fd968c5c5556ccd97ebcaba4be90"}
Oct 30 21:34:38.010517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.010126    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerStarted","Data":"9d9dc3d7e530771c25bde57bf8b60e5befbec6e33f9d653e8787ae4924ba90e5"}
Oct 30 21:34:38.010517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.010152    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerStarted","Data":"dc6b0437266041a647cfdef0c5a514eb13e30703219439b5b7a243e241c08ceb"}
Oct 30 21:34:38.010517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.010176    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerStarted","Data":"7b453ffe1868876225fbd437aa846771ab2b0ae827d4e81a88f51d5b42e5163e"}
Oct 30 21:34:38.010517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.010199    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96","Type":"ContainerStarted","Data":"0964a0ac2a947131edb4ec47675a3c36e5cfec3c4a30d684adf9eb9493c9a6b1"}
Oct 30 21:34:38.200236 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.200210    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:38.204297 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.204276    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:38.204449 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.204369    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:38.208888 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.208868    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:38.311952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.311923    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312124 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312001    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312124 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312031    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312124 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312075    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\" (UniqueName: \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312124 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312103    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312127    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312169    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312201    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\") pod \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\" (UID: \"e93dfe0f-83db-4682-a489-67ca3f81ee3b\") "
Oct 30 21:34:38.312340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312207    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:38.312587 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312418    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:38.312641 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312612    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:38.312641 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312618    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:38.312819 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312800    5561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.312952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.312929    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:38.313122 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.313058    5561 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e93dfe0f-83db-4682-a489-67ca3f81ee3b-tmp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.314499 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.314475    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:38.314855 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.314836    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:38.314904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.314837    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj" pod "e93dfe0f-83db-4682-a489-67ca3f81ee3b" (UID: "e93dfe0f-83db-4682-a489-67ca3f81ee3b"). InnerVolumeSpecName "kube-api-access-pxxzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:38.318726 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.318676    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.3069840360000002 podStartE2EDuration="3.318660351s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.676315ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:38.152740395 +0000 UTC m=+26.430433867" watchObservedRunningTime="2026-10-30 21:34:38.318660351 +0000 UTC m=+26.596353822"
Oct 30 21:34:38.413859 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.413834    5561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-serving-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.413859 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.413856    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxxzj\" (UniqueName: \"kubernetes.io/projected/e93dfe0f-83db-4682-a489-67ca3f81ee3b-kube-api-access-pxxzj\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.413982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.413866    5561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-oauth-serving-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.413982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.413877    5561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-trusted-ca-bundle\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.413982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.413886    5561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e93dfe0f-83db-4682-a489-67ca3f81ee3b-console-oauth-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.413982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:38.413896    5561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e93dfe0f-83db-4682-a489-67ca3f81ee3b-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.019518 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.019488    5561 generic.go:356] "Generic (PLEG): container finished" podID="e93dfe0f-83db-4682-a489-67ca3f81ee3b" containerID="ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786" exitCode=0
Oct 30 21:34:39.019917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.019543    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc8c7745-jf6t2"
Oct 30 21:34:39.019917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.019571    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc8c7745-jf6t2" event={"ID":"e93dfe0f-83db-4682-a489-67ca3f81ee3b","Type":"ContainerDied","Data":"ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786"}
Oct 30 21:34:39.019917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.019635    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc8c7745-jf6t2" event={"ID":"e93dfe0f-83db-4682-a489-67ca3f81ee3b","Type":"ContainerDied","Data":"f20eb2776a9e632e63d6464ea11d7fb031bd780917062c6a70f97577fc46f625"}
Oct 30 21:34:39.019917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.019664    5561 scope.go:122] "RemoveContainer" containerID="ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786"
Oct 30 21:34:39.030492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.030465    5561 scope.go:122] "RemoveContainer" containerID="ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786"
Oct 30 21:34:39.030762 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:39.030736    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786\": container with ID starting with ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786 not found: ID does not exist" containerID="ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786"
Oct 30 21:34:39.030835 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.030775    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786"} err="failed to get container status \"ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786\": rpc error: code = NotFound desc = could not find container \"ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786\": container with ID starting with ec66164d12894da3e459516455d717264521c67ca4a3559e38695f3bdbfcb786 not found: ID does not exist"
Oct 30 21:34:39.031246 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.031226    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c8d4d4b84-d8b7c"
Oct 30 21:34:39.183111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.183078    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bc8c7745-jf6t2"]
Oct 30 21:34:39.233430 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.233400    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86bc8c7745-jf6t2"]
Oct 30 21:34:39.336453 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.336395    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8664b77bf7-dmlx7"]
Oct 30 21:34:39.843258 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:39.843221    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e93dfe0f-83db-4682-a489-67ca3f81ee3b" path="/var/lib/kubelet/pods/e93dfe0f-83db-4682-a489-67ca3f81ee3b/volumes"
Oct 30 21:34:41.082238 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.082201    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-bfb94f7fb-rc2vj" podUID="170fa4df-5ce6-4243-b30a-7b2c91a733ce" containerName="console" containerID="cri-o://ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94" gracePeriod=15
Oct 30 21:34:41.326138 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.326117    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:41.439969 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.439939    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.439987    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440023    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440090    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440277 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440138    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440277 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440171    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440277 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440208    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\" (UniqueName: \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440277 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440243    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\" (UniqueName: \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\") pod \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\" (UID: \"170fa4df-5ce6-4243-b30a-7b2c91a733ce\") "
Oct 30 21:34:41.440502 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440323    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:41.440612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440583    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:41.440612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440599    5561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-oauth-serving-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:41.440805 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440619    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:41.440805 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440647    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:41.440805 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.440769    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:41.442844 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.442817    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "kube-api-access-h2z7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:41.442940 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.442858    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:41.442993 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.442938    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config" pod "170fa4df-5ce6-4243-b30a-7b2c91a733ce" (UID: "170fa4df-5ce6-4243-b30a-7b2c91a733ce"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:41.541784 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.541759    5561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:41.541784 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.541781    5561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:41.541904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.541792    5561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-oauth-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:41.541904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.541802    5561 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/170fa4df-5ce6-4243-b30a-7b2c91a733ce-tmp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:41.541904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.541812    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2z7p\" (UniqueName: \"kubernetes.io/projected/170fa4df-5ce6-4243-b30a-7b2c91a733ce-kube-api-access-h2z7p\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:41.541904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.541822    5561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/170fa4df-5ce6-4243-b30a-7b2c91a733ce-trusted-ca-bundle\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:41.541904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:41.541831    5561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/170fa4df-5ce6-4243-b30a-7b2c91a733ce-console-serving-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:42.031524 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.031488    5561 generic.go:356] "Generic (PLEG): container finished" podID="170fa4df-5ce6-4243-b30a-7b2c91a733ce" containerID="ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94" exitCode=0
Oct 30 21:34:42.031667 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.031533    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb94f7fb-rc2vj"
Oct 30 21:34:42.031667 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.031566    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb94f7fb-rc2vj" event={"ID":"170fa4df-5ce6-4243-b30a-7b2c91a733ce","Type":"ContainerDied","Data":"ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94"}
Oct 30 21:34:42.031667 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.031615    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb94f7fb-rc2vj" event={"ID":"170fa4df-5ce6-4243-b30a-7b2c91a733ce","Type":"ContainerDied","Data":"ec90556b38e0cbffdea98f4d7450d51fa6f5310d34331ef0bfd22f8ca48f9fe1"}
Oct 30 21:34:42.031667 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.031636    5561 scope.go:122] "RemoveContainer" containerID="ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94"
Oct 30 21:34:42.039197 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.039177    5561 scope.go:122] "RemoveContainer" containerID="ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94"
Oct 30 21:34:42.039501 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:42.039481    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94\": container with ID starting with ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94 not found: ID does not exist" containerID="ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94"
Oct 30 21:34:42.039567 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.039514    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94"} err="failed to get container status \"ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94\": rpc error: code = NotFound desc = could not find container \"ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94\": container with ID starting with ec0bc8c2d862ac9b7edcf19deae4bbdeda96fe4c8a7052c2b29108127e422a94 not found: ID does not exist"
Oct 30 21:34:42.196412 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.196381    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bfb94f7fb-rc2vj"]
Oct 30 21:34:42.246725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.246701    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bfb94f7fb-rc2vj"]
Oct 30 21:34:42.808600 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.808572    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:42.812249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:42.812225    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-66b4c7697-8m46d"
Oct 30 21:34:43.842280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:43.842245    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="170fa4df-5ce6-4243-b30a-7b2c91a733ce" path="/var/lib/kubelet/pods/170fa4df-5ce6-4243-b30a-7b2c91a733ce/volumes"
Oct 30 21:34:49.185475 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:34:49.185443    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b6gtf"
Oct 30 21:34:52.980436 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:34:52.980401    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:35:04.359182 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.359117    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-8664b77bf7-dmlx7" podUID="42452a26-0fab-4ba7-acfb-c6781234a18f" containerName="console" containerID="cri-o://93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978" gracePeriod=15
Oct 30 21:35:04.597334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.597281    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:35:04.697720 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697694    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-serving-cert\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.697851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697731    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-console-config\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-console-config\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.697851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697783    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42452a26-0fab-4ba7-acfb-c6781234a18f-kube-api-access-7mj9b\" (UniqueName: \"kubernetes.io/projected/42452a26-0fab-4ba7-acfb-c6781234a18f-kube-api-access-7mj9b\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.697851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697800    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-oauth-config\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.697851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697823    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-oauth-serving-cert\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.698064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697854    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42452a26-0fab-4ba7-acfb-c6781234a18f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/42452a26-0fab-4ba7-acfb-c6781234a18f-tmp\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.698064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697919    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-service-ca\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-service-ca\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.698064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.697958    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-trusted-ca-bundle\") pod \"42452a26-0fab-4ba7-acfb-c6781234a18f\" (UID: \"42452a26-0fab-4ba7-acfb-c6781234a18f\") "
Oct 30 21:35:04.698218 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.698157    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42452a26-0fab-4ba7-acfb-c6781234a18f-tmp" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:35:04.698274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.698222    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-console-config" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:04.698274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.698257    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-oauth-serving-cert" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:04.698570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.698513    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-service-ca" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:04.698570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.698537    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-trusted-ca-bundle" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:04.699874 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.699853    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42452a26-0fab-4ba7-acfb-c6781234a18f-kube-api-access-7mj9b" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "kube-api-access-7mj9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:04.700214 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.700190    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-oauth-config" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:04.700214 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.700203    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-serving-cert" pod "42452a26-0fab-4ba7-acfb-c6781234a18f" (UID: "42452a26-0fab-4ba7-acfb-c6781234a18f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:35:04.798982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.798950    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mj9b\" (UniqueName: \"kubernetes.io/projected/42452a26-0fab-4ba7-acfb-c6781234a18f-kube-api-access-7mj9b\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.798982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.798974    5561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-oauth-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.798982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.798985    5561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-oauth-serving-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.799196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.798994    5561 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42452a26-0fab-4ba7-acfb-c6781234a18f-tmp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.799196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.799004    5561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.799196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.799012    5561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-trusted-ca-bundle\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.799196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.799022    5561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42452a26-0fab-4ba7-acfb-c6781234a18f-console-serving-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:04.799196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:04.799032    5561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42452a26-0fab-4ba7-acfb-c6781234a18f-console-config\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:35:05.113651 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.113578    5561 generic.go:356] "Generic (PLEG): container finished" podID="42452a26-0fab-4ba7-acfb-c6781234a18f" containerID="93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978" exitCode=0
Oct 30 21:35:05.113651 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.113624    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664b77bf7-dmlx7"
Oct 30 21:35:05.113804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.113661    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664b77bf7-dmlx7" event={"ID":"42452a26-0fab-4ba7-acfb-c6781234a18f","Type":"ContainerDied","Data":"93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978"}
Oct 30 21:35:05.113804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.113713    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664b77bf7-dmlx7" event={"ID":"42452a26-0fab-4ba7-acfb-c6781234a18f","Type":"ContainerDied","Data":"2938f7d0de7013c46230ffe734fba30d12b371043308b2449ee7776dcdc9e785"}
Oct 30 21:35:05.113804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.113738    5561 scope.go:122] "RemoveContainer" containerID="93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978"
Oct 30 21:35:05.123946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.123924    5561 scope.go:122] "RemoveContainer" containerID="93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978"
Oct 30 21:35:05.124255 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:35:05.124231    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978\": container with ID starting with 93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978 not found: ID does not exist" containerID="93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978"
Oct 30 21:35:05.124342 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.124266    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978"} err="failed to get container status \"93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978\": rpc error: code = NotFound desc = could not find container \"93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978\": container with ID starting with 93dc6298b04e387ba6a3abf2a6c41da89d5fe676fe4e35e3528410ce44559978 not found: ID does not exist"
Oct 30 21:35:05.279768 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.279745    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8664b77bf7-dmlx7"]
Oct 30 21:35:05.330282 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.330259    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8664b77bf7-dmlx7"]
Oct 30 21:35:05.843088 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:35:05.843053    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42452a26-0fab-4ba7-acfb-c6781234a18f" path="/var/lib/kubelet/pods/42452a26-0fab-4ba7-acfb-c6781234a18f/volumes"
Oct 30 21:35:22.979848 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:35:22.979807    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:35:52.979326 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:35:52.979244    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:36:22.979660 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:36:22.979621    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:36:52.978853 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:36:52.978808    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:37:22.978973 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:37:22.978888    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:37:35.406615 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406577    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"]
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406848    5561 cpu_manager.go:414] "containerMap: removing container" podUID="e93dfe0f-83db-4682-a489-67ca3f81ee3b" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406860    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e93dfe0f-83db-4682-a489-67ca3f81ee3b" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406873    5561 cpu_manager.go:414] "containerMap: removing container" podUID="42452a26-0fab-4ba7-acfb-c6781234a18f" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406878    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42452a26-0fab-4ba7-acfb-c6781234a18f" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406895    5561 cpu_manager.go:414] "containerMap: removing container" podUID="170fa4df-5ce6-4243-b30a-7b2c91a733ce" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406901    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="170fa4df-5ce6-4243-b30a-7b2c91a733ce" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406954    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="170fa4df-5ce6-4243-b30a-7b2c91a733ce" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406964    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="e93dfe0f-83db-4682-a489-67ca3f81ee3b" containerName="console"
Oct 30 21:37:35.406989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.406971    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="42452a26-0fab-4ba7-acfb-c6781234a18f" containerName="console"
Oct 30 21:37:35.408820 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.408804    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"]
Oct 30 21:37:35.408944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.408926    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.410611 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.410595    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.465227 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.465199    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:37:35.465399 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.465382    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:37:35.465482 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.465415    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-6d7kc\""
Oct 30 21:37:35.485936 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.485913    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.486004 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.485940    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvh9k\" (UniqueName: \"kubernetes.io/projected/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-kube-api-access-nvh9k\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.486004 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.485964    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.486086 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.486007    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.486086 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.486073    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwqdk\" (UniqueName: \"kubernetes.io/projected/abe57181-2303-4e58-83bc-a4b433d4dd52-kube-api-access-kwqdk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.486155 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.486117    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.515183 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.515160    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-9fs5n\""
Oct 30 21:37:35.529654 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.529631    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"]
Oct 30 21:37:35.530214 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.530190    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"]
Oct 30 21:37:35.586758 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.586735    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.586895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.586763    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvh9k\" (UniqueName: \"kubernetes.io/projected/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-kube-api-access-nvh9k\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.586895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.586787    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.586895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.586803    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.586895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.586823    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwqdk\" (UniqueName: \"kubernetes.io/projected/abe57181-2303-4e58-83bc-a4b433d4dd52-kube-api-access-kwqdk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.587115 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.586890    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.587226 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.587208    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.587263 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.587208    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.587263 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.587208    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.587323 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.587259    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.653781 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.653760    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwqdk\" (UniqueName: \"kubernetes.io/projected/abe57181-2303-4e58-83bc-a4b433d4dd52-kube-api-access-kwqdk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.658230 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.658165    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvh9k\" (UniqueName: \"kubernetes.io/projected/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-kube-api-access-nvh9k\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.721625 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.721603    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:35.726201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.726185    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:35.921593 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.921520    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"]
Oct 30 21:37:35.923694 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.923672    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:35.979202 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.979182    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:37:35.979485 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.979468    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:37:35.980603 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.980584    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:37:35.980834 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.980820    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:37:35.980897 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.980821    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:37:35.981024 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.981014    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4tfh8\""
Oct 30 21:37:35.982068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.982053    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:37:35.988690 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.988668    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"]
Oct 30 21:37:35.989114 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.989094    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:35.989165 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.989127    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:35.989165 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.989149    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:35.989285 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.989265    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svzfq\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-kube-api-access-svzfq\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:35.989326 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.989304    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:35.989458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.989437    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:35.989532 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:35.989516    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa3f27-d6e7-4cab-95b0-330c75c088df-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.090771 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.090737    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa3f27-d6e7-4cab-95b0-330c75c088df-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.090953 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.090780    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.090953 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.090809    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.090953 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.090834    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.090953 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.090868    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svzfq\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-kube-api-access-svzfq\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.090953 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.090905    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.090953 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.090954    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.091564 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.091537    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.093234 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.093211    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.093234 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.093226    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.093384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.093251    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.093384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.093226    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa3f27-d6e7-4cab-95b0-330c75c088df-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.098396 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.098376    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"]
Oct 30 21:37:36.100460 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:37:36.100437    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf46856_00c8_4d86_ba27_f5e1a4c1506f.slice/crio-1719fdfd40662c3ea97a7044f166d09daed015c31098d38b55da1c4922e4291f WatchSource:0}: Error finding container 1719fdfd40662c3ea97a7044f166d09daed015c31098d38b55da1c4922e4291f: Status 404 returned error can't find the container with id 1719fdfd40662c3ea97a7044f166d09daed015c31098d38b55da1c4922e4291f
Oct 30 21:37:36.102232 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.102217    5561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:37:36.143509 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.143490    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"]
Oct 30 21:37:36.145252 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:37:36.145228    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe57181_2303_4e58_83bc_a4b433d4dd52.slice/crio-1e7e22ddaa1bc421f1010bd6e6e1af4eb979cedeca7e81ce92b4814130186dab WatchSource:0}: Error finding container 1e7e22ddaa1bc421f1010bd6e6e1af4eb979cedeca7e81ce92b4814130186dab: Status 404 returned error can't find the container with id 1e7e22ddaa1bc421f1010bd6e6e1af4eb979cedeca7e81ce92b4814130186dab
Oct 30 21:37:36.150498 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.150477    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svzfq\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-kube-api-access-svzfq\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.150586 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.150501    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-mhjgp\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.233834 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.233781    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:36.602031 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.601851    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"]
Oct 30 21:37:36.604324 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:37:36.604296    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6fa3f27_d6e7_4cab_95b0_330c75c088df.slice/crio-88e97655b66e6ac1e190e635a8d2fadef588e8747d6d2d39de2f3cfcd14ecc27 WatchSource:0}: Error finding container 88e97655b66e6ac1e190e635a8d2fadef588e8747d6d2d39de2f3cfcd14ecc27: Status 404 returned error can't find the container with id 88e97655b66e6ac1e190e635a8d2fadef588e8747d6d2d39de2f3cfcd14ecc27
Oct 30 21:37:36.612225 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.612204    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr" event={"ID":"abe57181-2303-4e58-83bc-a4b433d4dd52","Type":"ContainerStarted","Data":"1e7e22ddaa1bc421f1010bd6e6e1af4eb979cedeca7e81ce92b4814130186dab"}
Oct 30 21:37:36.613388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.613344    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" event={"ID":"f6fa3f27-d6e7-4cab-95b0-330c75c088df","Type":"ContainerStarted","Data":"88e97655b66e6ac1e190e635a8d2fadef588e8747d6d2d39de2f3cfcd14ecc27"}
Oct 30 21:37:36.614439 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:36.614419    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd" event={"ID":"eaf46856-00c8-4d86-ba27-f5e1a4c1506f","Type":"ContainerStarted","Data":"1719fdfd40662c3ea97a7044f166d09daed015c31098d38b55da1c4922e4291f"}
Oct 30 21:37:40.419193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:40.418799    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:37:40.419193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:40.418879    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:37:40.633781 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:40.633745    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" event={"ID":"f6fa3f27-d6e7-4cab-95b0-330c75c088df","Type":"ContainerStarted","Data":"8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a"}
Oct 30 21:37:40.634030 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:40.634002    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:40.764235 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:40.764141    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" podStartSLOduration=1.9553759689999999 podStartE2EDuration="5.764115906s" totalImagesPullingTime="3.808739937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:40.76220534 +0000 UTC m=+209.039898812" watchObservedRunningTime="2026-10-30 21:37:40.764115906 +0000 UTC m=+209.041809378"
Oct 30 21:37:41.646145 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:41.646112    5561 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-mhjgp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:37:41.650843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:41.650521    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:37:43.652697 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:43.652663    5561 generic.go:356] "Generic (PLEG): container finished" podID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerID="ba6950ea7a079e7d75248810f90c6fad87b2d11ac53706314b27ca7a7af71d6b" exitCode=0
Oct 30 21:37:43.653055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:43.652747    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr" event={"ID":"abe57181-2303-4e58-83bc-a4b433d4dd52","Type":"ContainerDied","Data":"ba6950ea7a079e7d75248810f90c6fad87b2d11ac53706314b27ca7a7af71d6b"}
Oct 30 21:37:44.645119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:44.645084    5561 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-mhjgp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:37:44.645280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:44.645138    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:37:44.657404 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:44.657376    5561 generic.go:356] "Generic (PLEG): container finished" podID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerID="61d871ef67ad71854042fb5eb7e94f1826fff503b67050ed3c568b70138d11fe" exitCode=0
Oct 30 21:37:44.657757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:44.657463    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd" event={"ID":"eaf46856-00c8-4d86-ba27-f5e1a4c1506f","Type":"ContainerDied","Data":"61d871ef67ad71854042fb5eb7e94f1826fff503b67050ed3c568b70138d11fe"}
Oct 30 21:37:47.644949 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:47.644909    5561 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-mhjgp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:37:47.645346 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:47.644968    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:37:47.672546 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:47.672513    5561 generic.go:356] "Generic (PLEG): container finished" podID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerID="c1ff4441e75844071594dbcb1979c60707b11e7f431a4579f8a2c9bffcc0daa9" exitCode=0
Oct 30 21:37:47.673183 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:47.673143    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr" event={"ID":"abe57181-2303-4e58-83bc-a4b433d4dd52","Type":"ContainerDied","Data":"c1ff4441e75844071594dbcb1979c60707b11e7f431a4579f8a2c9bffcc0daa9"}
Oct 30 21:37:48.678638 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:48.678601    5561 generic.go:356] "Generic (PLEG): container finished" podID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerID="cfba1ee58f2cc396d0cdf2ecf6741bd99b0d4e0b062a01fde8b75a402671c6ce" exitCode=0
Oct 30 21:37:48.679098 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:48.678712    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd" event={"ID":"eaf46856-00c8-4d86-ba27-f5e1a4c1506f","Type":"ContainerDied","Data":"cfba1ee58f2cc396d0cdf2ecf6741bd99b0d4e0b062a01fde8b75a402671c6ce"}
Oct 30 21:37:50.645420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:50.645376    5561 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-mhjgp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:37:50.645838 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:50.645435    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:37:52.989562 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:37:52.989492    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:37:53.645841 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:53.645805    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:37:55.708599 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:55.708565    5561 generic.go:356] "Generic (PLEG): container finished" podID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerID="8333dae93e4b04e70fee42cf6ca4c67aa7b19b066b1f242f7187d687f2dae8b0" exitCode=0
Oct 30 21:37:55.708990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:55.708646    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr" event={"ID":"abe57181-2303-4e58-83bc-a4b433d4dd52","Type":"ContainerDied","Data":"8333dae93e4b04e70fee42cf6ca4c67aa7b19b066b1f242f7187d687f2dae8b0"}
Oct 30 21:37:56.714515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.714432    5561 generic.go:356] "Generic (PLEG): container finished" podID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerID="6fbaa55b07a3e4e4dfed02138cff42b542e438fb2fff438282f0d345444fad02" exitCode=0
Oct 30 21:37:56.714515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.714475    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd" event={"ID":"eaf46856-00c8-4d86-ba27-f5e1a4c1506f","Type":"ContainerDied","Data":"6fbaa55b07a3e4e4dfed02138cff42b542e438fb2fff438282f0d345444fad02"}
Oct 30 21:37:56.828601 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.828577    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:56.974660 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.974596    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-bundle\") pod \"abe57181-2303-4e58-83bc-a4b433d4dd52\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") "
Oct 30 21:37:56.974660 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.974640    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abe57181-2303-4e58-83bc-a4b433d4dd52-kube-api-access-kwqdk\" (UniqueName: \"kubernetes.io/projected/abe57181-2303-4e58-83bc-a4b433d4dd52-kube-api-access-kwqdk\") pod \"abe57181-2303-4e58-83bc-a4b433d4dd52\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") "
Oct 30 21:37:56.974827 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.974677    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-util\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-util\") pod \"abe57181-2303-4e58-83bc-a4b433d4dd52\" (UID: \"abe57181-2303-4e58-83bc-a4b433d4dd52\") "
Oct 30 21:37:56.975042 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.975015    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-bundle" pod "abe57181-2303-4e58-83bc-a4b433d4dd52" (UID: "abe57181-2303-4e58-83bc-a4b433d4dd52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:56.976875 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.976855    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe57181-2303-4e58-83bc-a4b433d4dd52-kube-api-access-kwqdk" pod "abe57181-2303-4e58-83bc-a4b433d4dd52" (UID: "abe57181-2303-4e58-83bc-a4b433d4dd52"). InnerVolumeSpecName "kube-api-access-kwqdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:56.979824 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:56.979802    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-util" pod "abe57181-2303-4e58-83bc-a4b433d4dd52" (UID: "abe57181-2303-4e58-83bc-a4b433d4dd52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:57.075780 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.075757    5561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-util\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:57.075780 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.075779    5561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe57181-2303-4e58-83bc-a4b433d4dd52-bundle\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:57.075929 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.075788    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwqdk\" (UniqueName: \"kubernetes.io/projected/abe57181-2303-4e58-83bc-a4b433d4dd52-kube-api-access-kwqdk\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:57.719555 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.719523    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr"
Oct 30 21:37:57.719555 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.719526    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e64464hcr" event={"ID":"abe57181-2303-4e58-83bc-a4b433d4dd52","Type":"ContainerDied","Data":"1e7e22ddaa1bc421f1010bd6e6e1af4eb979cedeca7e81ce92b4814130186dab"}
Oct 30 21:37:57.720003 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.719575    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e7e22ddaa1bc421f1010bd6e6e1af4eb979cedeca7e81ce92b4814130186dab"
Oct 30 21:37:57.849127 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.849098    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:57.983156 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.983084    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-bundle\") pod \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") "
Oct 30 21:37:57.983156 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.983121    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-util\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-util\") pod \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") "
Oct 30 21:37:57.983381 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.983185    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-kube-api-access-nvh9k\" (UniqueName: \"kubernetes.io/projected/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-kube-api-access-nvh9k\") pod \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\" (UID: \"eaf46856-00c8-4d86-ba27-f5e1a4c1506f\") "
Oct 30 21:37:57.983690 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.983665    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-bundle" pod "eaf46856-00c8-4d86-ba27-f5e1a4c1506f" (UID: "eaf46856-00c8-4d86-ba27-f5e1a4c1506f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:57.985241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.985211    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-kube-api-access-nvh9k" pod "eaf46856-00c8-4d86-ba27-f5e1a4c1506f" (UID: "eaf46856-00c8-4d86-ba27-f5e1a4c1506f"). InnerVolumeSpecName "kube-api-access-nvh9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:57.987727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:57.987706    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-util" pod "eaf46856-00c8-4d86-ba27-f5e1a4c1506f" (UID: "eaf46856-00c8-4d86-ba27-f5e1a4c1506f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:58.084307 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:58.084281    5561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-bundle\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:58.084307 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:58.084302    5561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-util\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:58.084456 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:58.084312    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvh9k\" (UniqueName: \"kubernetes.io/projected/eaf46856-00c8-4d86-ba27-f5e1a4c1506f-kube-api-access-nvh9k\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:37:58.725165 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:58.725132    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd"
Oct 30 21:37:58.725578 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:58.725134    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ccvlfd" event={"ID":"eaf46856-00c8-4d86-ba27-f5e1a4c1506f","Type":"ContainerDied","Data":"1719fdfd40662c3ea97a7044f166d09daed015c31098d38b55da1c4922e4291f"}
Oct 30 21:37:58.725578 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:37:58.725259    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1719fdfd40662c3ea97a7044f166d09daed015c31098d38b55da1c4922e4291f"
Oct 30 21:38:17.664095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664060    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"]
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664306    5561 cpu_manager.go:414] "containerMap: removing container" podUID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerName="extract"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664317    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerName="extract"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664336    5561 cpu_manager.go:414] "containerMap: removing container" podUID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerName="pull"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664341    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerName="pull"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664361    5561 cpu_manager.go:414] "containerMap: removing container" podUID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerName="extract"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664366    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerName="extract"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664379    5561 cpu_manager.go:414] "containerMap: removing container" podUID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerName="util"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664386    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerName="util"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664397    5561 cpu_manager.go:414] "containerMap: removing container" podUID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerName="pull"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664402    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerName="pull"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664407    5561 cpu_manager.go:414] "containerMap: removing container" podUID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerName="util"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664412    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerName="util"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664455    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="eaf46856-00c8-4d86-ba27-f5e1a4c1506f" containerName="extract"
Oct 30 21:38:17.665384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.664465    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="abe57181-2303-4e58-83bc-a4b433d4dd52" containerName="extract"
Oct 30 21:38:17.665951 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.665935    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.774501 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.774461    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"]
Oct 30 21:38:17.816363 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.816321    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.816510 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.816369    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-tmp\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.816510 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.816391    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-manifests\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.816612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.816538    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.816612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.816575    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2p5r\" (UniqueName: \"kubernetes.io/projected/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-kube-api-access-d2p5r\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.816697 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.816615    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.823005 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.822975    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:38:17.823127 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.823069    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:38:17.823127 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.822984    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:38:17.823303 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.823281    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:38:17.823489 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.823297    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-xkqbq\""
Oct 30 21:38:17.823603 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.823382    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:38:17.917344 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.917280    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.917344 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.917308    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2p5r\" (UniqueName: \"kubernetes.io/projected/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-kube-api-access-d2p5r\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.917344 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.917337    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.917584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.917382    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.917584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.917505    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-tmp\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.917584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.917542    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-manifests\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.917961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.917934    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-manifests\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.919711 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.919684    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.919711 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.919697    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-tmp\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.919987 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.919795    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.919987 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.919889    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:17.977582 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:17.977553    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2p5r\" (UniqueName: \"kubernetes.io/projected/b6d6a0ac-8bca-4fa8-8666-f79db90bd428-kube-api-access-d2p5r\") pod \"opendatahub-operator-controller-manager-7c746f769b-jlpzt\" (UID: \"b6d6a0ac-8bca-4fa8-8666-f79db90bd428\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:18.277458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:18.277386    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:18.814151 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:18.814121    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"]
Oct 30 21:38:18.817658 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:38:18.817634    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d6a0ac_8bca_4fa8_8666_f79db90bd428.slice/crio-918e70b55854fc448a1bd7607e4380d389975919789ee82ed27fcd16641a0218 WatchSource:0}: Error finding container 918e70b55854fc448a1bd7607e4380d389975919789ee82ed27fcd16641a0218: Status 404 returned error can't find the container with id 918e70b55854fc448a1bd7607e4380d389975919789ee82ed27fcd16641a0218
Oct 30 21:38:19.799440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:19.799392    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt" event={"ID":"b6d6a0ac-8bca-4fa8-8666-f79db90bd428","Type":"ContainerStarted","Data":"918e70b55854fc448a1bd7607e4380d389975919789ee82ed27fcd16641a0218"}
Oct 30 21:38:22.814170 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:22.814087    5561 generic.go:356] "Generic (PLEG): container finished" podID="b6d6a0ac-8bca-4fa8-8666-f79db90bd428" containerID="30cbb87c0619089b09f51c4c21fbc1a568d3a259ecfaa8aa2d7cf95abf80ea54" exitCode=0
Oct 30 21:38:22.814170 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:22.814144    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt" event={"ID":"b6d6a0ac-8bca-4fa8-8666-f79db90bd428","Type":"ContainerDied","Data":"30cbb87c0619089b09f51c4c21fbc1a568d3a259ecfaa8aa2d7cf95abf80ea54"}
Oct 30 21:38:22.980109 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:38:22.980072    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:38:24.823210 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:24.823171    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt" event={"ID":"b6d6a0ac-8bca-4fa8-8666-f79db90bd428","Type":"ContainerStarted","Data":"92e8adb4828cee809cf790014de4632f93c1b357ee4de37e9574e31cceccdcdb"}
Oct 30 21:38:24.823662 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:24.823283    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:25.114840 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:25.114744    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt" podStartSLOduration=3.542300316 podStartE2EDuration="8.114732422s" totalImagesPullingTime="4.161502579s" totalInitContainerRuntime="410.929527ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:25.063494592 +0000 UTC m=+253.341188065" watchObservedRunningTime="2026-10-30 21:38:25.114732422 +0000 UTC m=+253.392425893"
Oct 30 21:38:35.829028 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:35.828988    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-jlpzt"
Oct 30 21:38:44.829781 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.829728    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"]
Oct 30 21:38:44.833561 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.833537    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:44.887388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.887345    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dhlbq\""
Oct 30 21:38:44.926810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.926785    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g92nv\" (UniqueName: \"kubernetes.io/projected/66f4f206-f2fa-4191-9f7c-f8726e9a47dc-kube-api-access-g92nv\") pod \"cert-manager-cainjector-5c57bbd4fc-hw6sg\" (UID: \"66f4f206-f2fa-4191-9f7c-f8726e9a47dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:44.926938 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.926840    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66f4f206-f2fa-4191-9f7c-f8726e9a47dc-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-hw6sg\" (UID: \"66f4f206-f2fa-4191-9f7c-f8726e9a47dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:44.937110 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.937089    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 21:38:44.937110 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.937105    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 21:38:44.937892 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:44.937870    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"]
Oct 30 21:38:45.027780 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:45.027751    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66f4f206-f2fa-4191-9f7c-f8726e9a47dc-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-hw6sg\" (UID: \"66f4f206-f2fa-4191-9f7c-f8726e9a47dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:45.027911 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:45.027818    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g92nv\" (UniqueName: \"kubernetes.io/projected/66f4f206-f2fa-4191-9f7c-f8726e9a47dc-kube-api-access-g92nv\") pod \"cert-manager-cainjector-5c57bbd4fc-hw6sg\" (UID: \"66f4f206-f2fa-4191-9f7c-f8726e9a47dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:45.086084 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:45.086016    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66f4f206-f2fa-4191-9f7c-f8726e9a47dc-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-hw6sg\" (UID: \"66f4f206-f2fa-4191-9f7c-f8726e9a47dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:45.086243 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:45.086220    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g92nv\" (UniqueName: \"kubernetes.io/projected/66f4f206-f2fa-4191-9f7c-f8726e9a47dc-kube-api-access-g92nv\") pod \"cert-manager-cainjector-5c57bbd4fc-hw6sg\" (UID: \"66f4f206-f2fa-4191-9f7c-f8726e9a47dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:45.156251 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:45.156223    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"
Oct 30 21:38:45.522279 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:45.522254    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg"]
Oct 30 21:38:45.525079 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:38:45.525049    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66f4f206_f2fa_4191_9f7c_f8726e9a47dc.slice/crio-55fb7485a4826387d9a3245486dbb41eef21f104bfdb6b7b02cfb02e4ab37218 WatchSource:0}: Error finding container 55fb7485a4826387d9a3245486dbb41eef21f104bfdb6b7b02cfb02e4ab37218: Status 404 returned error can't find the container with id 55fb7485a4826387d9a3245486dbb41eef21f104bfdb6b7b02cfb02e4ab37218
Oct 30 21:38:45.896883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:45.896851    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg" event={"ID":"66f4f206-f2fa-4191-9f7c-f8726e9a47dc","Type":"ContainerStarted","Data":"55fb7485a4826387d9a3245486dbb41eef21f104bfdb6b7b02cfb02e4ab37218"}
Oct 30 21:38:52.923913 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:52.923876    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg" event={"ID":"66f4f206-f2fa-4191-9f7c-f8726e9a47dc","Type":"ContainerStarted","Data":"944bc66584bc2a7c65ad21bdd0273d0b4322f9a53875c0f9b5620d30745f7323"}
Oct 30 21:38:52.980705 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:38:52.980671    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:38:53.038069 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:38:53.038021    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-hw6sg" podStartSLOduration=2.172622271 podStartE2EDuration="9.038008597s" totalImagesPullingTime="6.865386326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:53.037413078 +0000 UTC m=+281.315106551" watchObservedRunningTime="2026-10-30 21:38:53.038008597 +0000 UTC m=+281.315702058"
Oct 30 21:39:11.876364 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:11.876329    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:39:11.876922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:11.876526    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:39:19.874776 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:19.874743    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"]
Oct 30 21:39:19.881526 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:19.881503    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:19.952654 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:19.952627    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"]
Oct 30 21:39:19.988894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:19.988871    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmxmc\" (UniqueName: \"kubernetes.io/projected/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-kube-api-access-dmxmc\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:19.989020 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:19.988903    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-tls-certs\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:19.989020 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:19.988933    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-tmp\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.003319 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.003283    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:39:20.003319 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.003287    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:39:20.003319 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.003295    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sg8hv\""
Oct 30 21:39:20.089876 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.089841    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-tmp\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.090023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.089932    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmxmc\" (UniqueName: \"kubernetes.io/projected/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-kube-api-access-dmxmc\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.090023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.089966    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-tls-certs\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.091890 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.091869    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-tmp\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.092289 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.092267    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-tls-certs\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.147080 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.147023    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmxmc\" (UniqueName: \"kubernetes.io/projected/b1c830f7-14d0-4673-83b5-cea2dd6a33c9-kube-api-access-dmxmc\") pod \"kube-auth-proxy-6c46d956fb-42ws7\" (UID: \"b1c830f7-14d0-4673-83b5-cea2dd6a33c9\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.193128 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.193091    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"
Oct 30 21:39:20.562604 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:20.562533    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7"]
Oct 30 21:39:20.565756 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:39:20.565728    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c830f7_14d0_4673_83b5_cea2dd6a33c9.slice/crio-72c5b2425cb86039c34ee2878ec6a44d9ec0dcb13f9033999f2e7e8beb2cced5 WatchSource:0}: Error finding container 72c5b2425cb86039c34ee2878ec6a44d9ec0dcb13f9033999f2e7e8beb2cced5: Status 404 returned error can't find the container with id 72c5b2425cb86039c34ee2878ec6a44d9ec0dcb13f9033999f2e7e8beb2cced5
Oct 30 21:39:21.027128 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:21.027088    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7" event={"ID":"b1c830f7-14d0-4673-83b5-cea2dd6a33c9","Type":"ContainerStarted","Data":"72c5b2425cb86039c34ee2878ec6a44d9ec0dcb13f9033999f2e7e8beb2cced5"}
Oct 30 21:39:22.990409 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:39:22.990367    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:39:24.040664 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:24.040630    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7" event={"ID":"b1c830f7-14d0-4673-83b5-cea2dd6a33c9","Type":"ContainerStarted","Data":"3eb9e03501c22bdee1980ae41a65a1ac076852d2f0c80ee93928adb710f661b7"}
Oct 30 21:39:26.060417 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.060342    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7" podStartSLOduration=4.395178527 podStartE2EDuration="7.06032915s" totalImagesPullingTime="2.665150623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:24.204040797 +0000 UTC m=+312.481734268" watchObservedRunningTime="2026-10-30 21:39:26.06032915 +0000 UTC m=+314.338022646"
Oct 30 21:39:26.061032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.061014    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:39:26.064495 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.064475    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.126059 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.126028    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:39:26.142922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.142888    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a450e568-8366-4af9-b113-0590beb38605-config-file\") pod \"limitador-limitador-7c9b87d574-t7q4m\" (UID: \"a450e568-8366-4af9-b113-0590beb38605\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.143056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.142945    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9h4r\" (UniqueName: \"kubernetes.io/projected/a450e568-8366-4af9-b113-0590beb38605-kube-api-access-t9h4r\") pod \"limitador-limitador-7c9b87d574-t7q4m\" (UID: \"a450e568-8366-4af9-b113-0590beb38605\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.175251 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.175220    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:39:26.175251 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.175220    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 21:39:26.176119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.176105    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hfdpc\""
Oct 30 21:39:26.176189 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.176171    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:39:26.243610 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.243587    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a450e568-8366-4af9-b113-0590beb38605-config-file\") pod \"limitador-limitador-7c9b87d574-t7q4m\" (UID: \"a450e568-8366-4af9-b113-0590beb38605\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.243756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.243666    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9h4r\" (UniqueName: \"kubernetes.io/projected/a450e568-8366-4af9-b113-0590beb38605-kube-api-access-t9h4r\") pod \"limitador-limitador-7c9b87d574-t7q4m\" (UID: \"a450e568-8366-4af9-b113-0590beb38605\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.244192 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.244172    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a450e568-8366-4af9-b113-0590beb38605-config-file\") pod \"limitador-limitador-7c9b87d574-t7q4m\" (UID: \"a450e568-8366-4af9-b113-0590beb38605\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.306111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.306079    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9h4r\" (UniqueName: \"kubernetes.io/projected/a450e568-8366-4af9-b113-0590beb38605-kube-api-access-t9h4r\") pod \"limitador-limitador-7c9b87d574-t7q4m\" (UID: \"a450e568-8366-4af9-b113-0590beb38605\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.362593 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.362529    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:39:26.376666 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.376643    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:26.765999 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:26.765964    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:39:26.768704 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:39:26.768679    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda450e568_8366_4af9_b113_0590beb38605.slice/crio-8fdbf33370e552d1c7683c1d1a133ffb1c1c2d12cdbe8d4557601a5726bc3451 WatchSource:0}: Error finding container 8fdbf33370e552d1c7683c1d1a133ffb1c1c2d12cdbe8d4557601a5726bc3451: Status 404 returned error can't find the container with id 8fdbf33370e552d1c7683c1d1a133ffb1c1c2d12cdbe8d4557601a5726bc3451
Oct 30 21:39:27.053835 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:27.053741    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m" event={"ID":"a450e568-8366-4af9-b113-0590beb38605","Type":"ContainerStarted","Data":"8fdbf33370e552d1c7683c1d1a133ffb1c1c2d12cdbe8d4557601a5726bc3451"}
Oct 30 21:39:30.067785 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:30.067702    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m" event={"ID":"a450e568-8366-4af9-b113-0590beb38605","Type":"ContainerStarted","Data":"cdfd4357988d1b0006dd578fbf6c567084ef2d85fb026eb306e0b7297469d0a8"}
Oct 30 21:39:30.068136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:30.067839    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:30.188982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:30.188932    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m" podStartSLOduration=1.301895002 podStartE2EDuration="4.188920938s" totalImagesPullingTime="2.887025936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:30.18861472 +0000 UTC m=+318.466308191" watchObservedRunningTime="2026-10-30 21:39:30.188920938 +0000 UTC m=+318.466614388"
Oct 30 21:39:41.073063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:39:41.073030    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"
Oct 30 21:39:52.979212 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:39:52.979173    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:40:16.835293 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.835262    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-gxhkw"]
Oct 30 21:40:16.844272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.844252    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:16.900046 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.900005    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:40:16.900173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.900115    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:40:16.900173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.900148    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:40:16.920621 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.920599    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhqth\" (UniqueName: \"kubernetes.io/projected/5db23168-3eca-4752-8384-01c03a78ce3e-kube-api-access-nhqth\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:16.920729 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.920637    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-metrics-certs\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:16.920729 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.920673    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-webhook-cert\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:16.950488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.950468    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7q82b\""
Oct 30 21:40:16.950584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.950492    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:40:16.950584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.950500    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:40:16.951663 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:16.951647    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-gxhkw"]
Oct 30 21:40:17.021866 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.021841    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-webhook-cert\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:17.021958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.021890    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhqth\" (UniqueName: \"kubernetes.io/projected/5db23168-3eca-4752-8384-01c03a78ce3e-kube-api-access-nhqth\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:17.022040 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.022020    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-metrics-certs\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:17.024238 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.024219    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-metrics-certs\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:17.024315 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.024241    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-webhook-cert\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:17.080393 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.080371    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhqth\" (UniqueName: \"kubernetes.io/projected/5db23168-3eca-4752-8384-01c03a78ce3e-kube-api-access-nhqth\") pod \"maas-controller-579cf7c6d-gxhkw\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") " pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:17.156660 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.156637    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:17.256641 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.256613    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7567487b8b-l7ph6"]
Oct 30 21:40:17.260002 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.259987    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.322595 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.322570    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7567487b8b-l7ph6"]
Oct 30 21:40:17.323663 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.323638    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-webhook-cert\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.323765 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.323673    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsc5x\" (UniqueName: \"kubernetes.io/projected/e7d7cc94-b718-4145-a7a1-c7c91876d78e-kube-api-access-wsc5x\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.323765 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.323700    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-metrics-certs\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.424981 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.424915    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-webhook-cert\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.424981 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.424947    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsc5x\" (UniqueName: \"kubernetes.io/projected/e7d7cc94-b718-4145-a7a1-c7c91876d78e-kube-api-access-wsc5x\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.424981 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.424975    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-metrics-certs\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.427304 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.427283    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-metrics-certs\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.427410 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.427322    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-webhook-cert\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.483296 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.483273    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsc5x\" (UniqueName: \"kubernetes.io/projected/e7d7cc94-b718-4145-a7a1-c7c91876d78e-kube-api-access-wsc5x\") pod \"maas-controller-7567487b8b-l7ph6\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") " pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.525575 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.525552    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-gxhkw"]
Oct 30 21:40:17.527543 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:40:17.527521    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db23168_3eca_4752_8384_01c03a78ce3e.slice/crio-7d9da9c13fc8d4bfc15acfc4be228630800d16110e41306b18c81e0b80ab3100 WatchSource:0}: Error finding container 7d9da9c13fc8d4bfc15acfc4be228630800d16110e41306b18c81e0b80ab3100: Status 404 returned error can't find the container with id 7d9da9c13fc8d4bfc15acfc4be228630800d16110e41306b18c81e0b80ab3100
Oct 30 21:40:17.572554 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.572528    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:17.944728 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:17.944703    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7567487b8b-l7ph6"]
Oct 30 21:40:17.946311 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:40:17.946286    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d7cc94_b718_4145_a7a1_c7c91876d78e.slice/crio-b2981565dc961d01c85bf06412a1c3592ef8bc01501c530ba77f98b463becd54 WatchSource:0}: Error finding container b2981565dc961d01c85bf06412a1c3592ef8bc01501c530ba77f98b463becd54: Status 404 returned error can't find the container with id b2981565dc961d01c85bf06412a1c3592ef8bc01501c530ba77f98b463becd54
Oct 30 21:40:18.263773 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:18.263673    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-gxhkw" event={"ID":"5db23168-3eca-4752-8384-01c03a78ce3e","Type":"ContainerStarted","Data":"7d9da9c13fc8d4bfc15acfc4be228630800d16110e41306b18c81e0b80ab3100"}
Oct 30 21:40:18.265035 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:18.265011    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-l7ph6" event={"ID":"e7d7cc94-b718-4145-a7a1-c7c91876d78e","Type":"ContainerStarted","Data":"b2981565dc961d01c85bf06412a1c3592ef8bc01501c530ba77f98b463becd54"}
Oct 30 21:40:21.282281 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:21.282244    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-gxhkw" event={"ID":"5db23168-3eca-4752-8384-01c03a78ce3e","Type":"ContainerStarted","Data":"72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86"}
Oct 30 21:40:21.282724 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:21.282336    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:21.283818 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:21.283797    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-l7ph6" event={"ID":"e7d7cc94-b718-4145-a7a1-c7c91876d78e","Type":"ContainerStarted","Data":"3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4"}
Oct 30 21:40:21.283933 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:21.283919    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:21.460612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:21.460555    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-gxhkw" podStartSLOduration=2.210131353 podStartE2EDuration="5.460541634s" totalImagesPullingTime="3.250410281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:21.456004626 +0000 UTC m=+369.733698100" watchObservedRunningTime="2026-10-30 21:40:21.460541634 +0000 UTC m=+369.738235082"
Oct 30 21:40:22.978697 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:22.978656    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:40:24.256344 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.256289    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7567487b8b-l7ph6" podStartSLOduration=4.134153672 podStartE2EDuration="7.256275141s" totalImagesPullingTime="3.122121469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:21.574750522 +0000 UTC m=+369.852443994" watchObservedRunningTime="2026-10-30 21:40:24.256275141 +0000 UTC m=+372.533968663"
Oct 30 21:40:24.257154 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.257135    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"]
Oct 30 21:40:24.260144 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.260129    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.291461 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.291436    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.291589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.291526    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.291589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.291553    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4zfm\" (UniqueName: \"kubernetes.io/projected/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-kube-api-access-b4zfm\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.291707 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.291689    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.313894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.313870    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 21:40:24.314020 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.313913    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 21:40:24.364774 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.364747    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 21:40:24.365180 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.365160    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-rm9g9\""
Oct 30 21:40:24.366165 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.366144    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"]
Oct 30 21:40:24.392831 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.392805    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.392929 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.392852    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.392929 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.392898    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.393043 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.392925    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4zfm\" (UniqueName: \"kubernetes.io/projected/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-kube-api-access-b4zfm\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.393043 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:24.392996    5561 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Oct 30 21:40:24.393140 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:24.393055    5561 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 21:40:24.393140 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:24.393065    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-proxy-source podName:6ae5b07c-0a6e-4ddb-aa67-741879d695b1 nodeName:}" failed. No retries permitted until 2026-10-30 21:40:24.893048997 +0000 UTC m=+373.170742446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-hrf8w" (UID: "6ae5b07c-0a6e-4ddb-aa67-741879d695b1") : configmap "usage-tenancy-proxy-source" not found
Oct 30 21:40:24.393140 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:24.393136    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-tls-cert podName:6ae5b07c-0a6e-4ddb-aa67-741879d695b1 nodeName:}" failed. No retries permitted until 2026-10-30 21:40:24.893119536 +0000 UTC m=+373.170812990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-hrf8w" (UID: "6ae5b07c-0a6e-4ddb-aa67-741879d695b1") : secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 21:40:24.393595 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.393573    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.454066 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.454039    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4zfm\" (UniqueName: \"kubernetes.io/projected/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-kube-api-access-b4zfm\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.897582 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.897550    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.897743 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.897607    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.898438 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.898407    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:24.899921 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:24.899900    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ae5b07c-0a6e-4ddb-aa67-741879d695b1-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hrf8w\" (UID: \"6ae5b07c-0a6e-4ddb-aa67-741879d695b1\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:25.171900 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:25.171824    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:25.545297 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:25.545217    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"]
Oct 30 21:40:25.549162 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:40:25.549129    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae5b07c_0a6e_4ddb_aa67_741879d695b1.slice/crio-43d83a59ecf7707cb8c7461270b3df936253d7a51886f10e88f0f292d53f679c WatchSource:0}: Error finding container 43d83a59ecf7707cb8c7461270b3df936253d7a51886f10e88f0f292d53f679c: Status 404 returned error can't find the container with id 43d83a59ecf7707cb8c7461270b3df936253d7a51886f10e88f0f292d53f679c
Oct 30 21:40:26.305080 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:26.305045    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w" event={"ID":"6ae5b07c-0a6e-4ddb-aa67-741879d695b1","Type":"ContainerStarted","Data":"43d83a59ecf7707cb8c7461270b3df936253d7a51886f10e88f0f292d53f679c"}
Oct 30 21:40:32.299386 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.299204    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:32.299386 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.299298    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:40:32.588244 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.588171    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-gxhkw"]
Oct 30 21:40:32.588422 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.588399    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-gxhkw" podUID="5db23168-3eca-4752-8384-01c03a78ce3e" containerName="manager" containerID="cri-o://72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86" gracePeriod=30
Oct 30 21:40:32.866702 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.866675    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:32.973036 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.973006    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-metrics-certs\") pod \"5db23168-3eca-4752-8384-01c03a78ce3e\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") "
Oct 30 21:40:32.973217 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.973134    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5db23168-3eca-4752-8384-01c03a78ce3e-kube-api-access-nhqth\" (UniqueName: \"kubernetes.io/projected/5db23168-3eca-4752-8384-01c03a78ce3e-kube-api-access-nhqth\") pod \"5db23168-3eca-4752-8384-01c03a78ce3e\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") "
Oct 30 21:40:32.973217 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.973180    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-webhook-cert\") pod \"5db23168-3eca-4752-8384-01c03a78ce3e\" (UID: \"5db23168-3eca-4752-8384-01c03a78ce3e\") "
Oct 30 21:40:32.975538 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.975508    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db23168-3eca-4752-8384-01c03a78ce3e-kube-api-access-nhqth" pod "5db23168-3eca-4752-8384-01c03a78ce3e" (UID: "5db23168-3eca-4752-8384-01c03a78ce3e"). InnerVolumeSpecName "kube-api-access-nhqth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:32.975878 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.975849    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-webhook-cert" pod "5db23168-3eca-4752-8384-01c03a78ce3e" (UID: "5db23168-3eca-4752-8384-01c03a78ce3e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:40:32.976413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:32.976378    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-metrics-certs" pod "5db23168-3eca-4752-8384-01c03a78ce3e" (UID: "5db23168-3eca-4752-8384-01c03a78ce3e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:40:33.074264 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.074224    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhqth\" (UniqueName: \"kubernetes.io/projected/5db23168-3eca-4752-8384-01c03a78ce3e-kube-api-access-nhqth\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.074264 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.074255    5561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-webhook-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.074487 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.074269    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5db23168-3eca-4752-8384-01c03a78ce3e-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:33.341572 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.341539    5561 generic.go:356] "Generic (PLEG): container finished" podID="5db23168-3eca-4752-8384-01c03a78ce3e" containerID="72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86" exitCode=0
Oct 30 21:40:33.342063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.341599    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-gxhkw"
Oct 30 21:40:33.342063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.341623    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-gxhkw" event={"ID":"5db23168-3eca-4752-8384-01c03a78ce3e","Type":"ContainerDied","Data":"72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86"}
Oct 30 21:40:33.342063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.341695    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-gxhkw" event={"ID":"5db23168-3eca-4752-8384-01c03a78ce3e","Type":"ContainerDied","Data":"7d9da9c13fc8d4bfc15acfc4be228630800d16110e41306b18c81e0b80ab3100"}
Oct 30 21:40:33.342063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.341726    5561 scope.go:122] "RemoveContainer" containerID="72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86"
Oct 30 21:40:33.351477 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.351451    5561 scope.go:122] "RemoveContainer" containerID="72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86"
Oct 30 21:40:33.351790 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:33.351768    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86\": container with ID starting with 72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86 not found: ID does not exist" containerID="72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86"
Oct 30 21:40:33.351889 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.351802    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86"} err="failed to get container status \"72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86\": rpc error: code = NotFound desc = could not find container \"72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86\": container with ID starting with 72d22684fcb162caa948e5bd768cec27f74964b26891284e2984120b94860f86 not found: ID does not exist"
Oct 30 21:40:33.509459 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.509423    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-gxhkw"]
Oct 30 21:40:33.515064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.515030    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-gxhkw"]
Oct 30 21:40:33.847317 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:33.847284    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5db23168-3eca-4752-8384-01c03a78ce3e" path="/var/lib/kubelet/pods/5db23168-3eca-4752-8384-01c03a78ce3e/volumes"
Oct 30 21:40:34.102597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.102515    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"]
Oct 30 21:40:34.102868 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.102855    5561 cpu_manager.go:414] "containerMap: removing container" podUID="5db23168-3eca-4752-8384-01c03a78ce3e" containerName="manager"
Oct 30 21:40:34.102924 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.102870    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5db23168-3eca-4752-8384-01c03a78ce3e" containerName="manager"
Oct 30 21:40:34.102965 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.102931    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="5db23168-3eca-4752-8384-01c03a78ce3e" containerName="manager"
Oct 30 21:40:34.119196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.119168    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.186048 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.186012    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.186201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.186067    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.186201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.186111    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.186201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.186154    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jlt4\" (UniqueName: \"kubernetes.io/projected/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-kube-api-access-8jlt4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.186201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.186193    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.186415 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.186219    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.186415 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.186296    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.237383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.237338    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"]
Oct 30 21:40:34.286863 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.286830    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.287044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.286886    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jlt4\" (UniqueName: \"kubernetes.io/projected/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-kube-api-access-8jlt4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.287044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.286925    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.287044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.286950    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.287044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.287018    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.287253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.287073    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.287253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.287101    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.288076 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.288045    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.290109 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.290086    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.290223 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.290179    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.290439 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.290412    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.290531 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.290460    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.346424 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.346393    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.352435 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.348072    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jlt4\" (UniqueName: \"kubernetes.io/projected/37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f-kube-api-access-8jlt4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-clbjj\" (UID: \"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.434437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.434408    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:34.871498 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.871435    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"]
Oct 30 21:40:34.874657 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:40:34.874630    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ba2ae5_f6f0_46b1_8315_aa0c5a801e3f.slice/crio-693534ccd514caee12e1f662981dd8113e22d443e98634435ba5ea4b5f80f65e WatchSource:0}: Error finding container 693534ccd514caee12e1f662981dd8113e22d443e98634435ba5ea4b5f80f65e: Status 404 returned error can't find the container with id 693534ccd514caee12e1f662981dd8113e22d443e98634435ba5ea4b5f80f65e
Oct 30 21:40:34.876875 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.876828    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:34.876968 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:34.876893    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:35.355320 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:35.355223    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj" event={"ID":"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f","Type":"ContainerStarted","Data":"b5f50aebbbd08f0d3fe1aec9c89a8f06fed0bd79ae30c3bd8fa4b3f21392ceb9"}
Oct 30 21:40:35.355320 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:35.355293    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj" event={"ID":"37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f","Type":"ContainerStarted","Data":"693534ccd514caee12e1f662981dd8113e22d443e98634435ba5ea4b5f80f65e"}
Oct 30 21:40:35.355320 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:35.355320    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:35.357338 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:35.357307    5561 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:40:35.357484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:35.357396    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj" podUID="37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:40:35.574072 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:35.574007    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj" podStartSLOduration=1.57398902 podStartE2EDuration="1.57398902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:35.523490764 +0000 UTC m=+383.801184232" watchObservedRunningTime="2026-10-30 21:40:35.57398902 +0000 UTC m=+383.851682495"
Oct 30 21:40:36.361270 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:36.361235    5561 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:40:36.361719 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:36.361313    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj" podUID="37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:40:38.370534 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:38.370497    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w" event={"ID":"6ae5b07c-0a6e-4ddb-aa67-741879d695b1","Type":"ContainerStarted","Data":"7095f448de2448736e48d83d776fecedff7b542ddb99df1c93c51a7b76cfdb60"}
Oct 30 21:40:38.370889 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:38.370617    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:38.491518 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:38.491459    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w" podStartSLOduration=1.789834873 podStartE2EDuration="14.491444554s" totalImagesPullingTime="12.701609681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:38.487579935 +0000 UTC m=+386.765273406" watchObservedRunningTime="2026-10-30 21:40:38.491444554 +0000 UTC m=+386.769138025"
Oct 30 21:40:39.361314 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.361285    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj"
Oct 30 21:40:39.509839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.509811    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"]
Oct 30 21:40:39.510206 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.509985    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery" containerID="cri-o://8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a" gracePeriod=30
Oct 30 21:40:39.755082 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.755056    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:40:39.833627 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.833595    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-kubeconfig\") pod \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") "
Oct 30 21:40:39.833793 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.833637    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6fa3f27-d6e7-4cab-95b0-330c75c088df-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa3f27-d6e7-4cab-95b0-330c75c088df-local-certs\") pod \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") "
Oct 30 21:40:39.833793 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.833674    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-token\") pod \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") "
Oct 30 21:40:39.833793 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.833707    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-ca-configmap\") pod \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") "
Oct 30 21:40:39.833793 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.833759    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-kube-api-access-svzfq\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-kube-api-access-svzfq\") pod \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") "
Oct 30 21:40:39.833793 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.833779    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-cacerts\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-cacerts\") pod \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") "
Oct 30 21:40:39.834051 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.833802    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-dns-cert\") pod \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\" (UID: \"f6fa3f27-d6e7-4cab-95b0-330c75c088df\") "
Oct 30 21:40:39.834222 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.834189    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-ca-configmap" pod "f6fa3f27-d6e7-4cab-95b0-330c75c088df" (UID: "f6fa3f27-d6e7-4cab-95b0-330c75c088df"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:39.836081 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.836055    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6fa3f27-d6e7-4cab-95b0-330c75c088df-local-certs" pod "f6fa3f27-d6e7-4cab-95b0-330c75c088df" (UID: "f6fa3f27-d6e7-4cab-95b0-330c75c088df"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:39.836194 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.836109    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-dns-cert" pod "f6fa3f27-d6e7-4cab-95b0-330c75c088df" (UID: "f6fa3f27-d6e7-4cab-95b0-330c75c088df"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:40:39.836254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.836229    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-kubeconfig" pod "f6fa3f27-d6e7-4cab-95b0-330c75c088df" (UID: "f6fa3f27-d6e7-4cab-95b0-330c75c088df"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:40:39.836314 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.836297    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-kube-api-access-svzfq" pod "f6fa3f27-d6e7-4cab-95b0-330c75c088df" (UID: "f6fa3f27-d6e7-4cab-95b0-330c75c088df"). InnerVolumeSpecName "kube-api-access-svzfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:39.836470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.836446    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-cacerts" pod "f6fa3f27-d6e7-4cab-95b0-330c75c088df" (UID: "f6fa3f27-d6e7-4cab-95b0-330c75c088df"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:40:39.836519 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.836476    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-token" pod "f6fa3f27-d6e7-4cab-95b0-330c75c088df" (UID: "f6fa3f27-d6e7-4cab-95b0-330c75c088df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:39.917597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.917526    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-2bsq2"]
Oct 30 21:40:39.917887 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.917875    5561 cpu_manager.go:414] "containerMap: removing container" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery"
Oct 30 21:40:39.917928 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.917889    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery"
Oct 30 21:40:39.917961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.917945    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerName="discovery"
Oct 30 21:40:39.921069 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.921050    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:39.936727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.936697    5561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-kubeconfig\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:39.936727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.936725    5561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa3f27-d6e7-4cab-95b0-330c75c088df-local-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:39.936909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.936738    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:39.936909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.936748    5561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-ca-configmap\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:39.936909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.936757    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svzfq\" (UniqueName: \"kubernetes.io/projected/f6fa3f27-d6e7-4cab-95b0-330c75c088df-kube-api-access-svzfq\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:39.936909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.936766    5561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-cacerts\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:39.936909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.936775    5561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6fa3f27-d6e7-4cab-95b0-330c75c088df-istio-csr-dns-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:40:39.974321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.974288    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 21:40:39.974492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.974332    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 21:40:39.974492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.974370    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qvhlb\""
Oct 30 21:40:39.980204 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:39.980175    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-2bsq2"]
Oct 30 21:40:40.037714 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.037681    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04432384-5c7b-4cf0-92eb-87d4beac19be-cert\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.037892 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.037741    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04432384-5c7b-4cf0-92eb-87d4beac19be-metrics-certs\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.037892 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.037827    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qpj9\" (UniqueName: \"kubernetes.io/projected/04432384-5c7b-4cf0-92eb-87d4beac19be-kube-api-access-6qpj9\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.138874 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.138838    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qpj9\" (UniqueName: \"kubernetes.io/projected/04432384-5c7b-4cf0-92eb-87d4beac19be-kube-api-access-6qpj9\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.139031 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.138909    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04432384-5c7b-4cf0-92eb-87d4beac19be-cert\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.139031 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.138972    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04432384-5c7b-4cf0-92eb-87d4beac19be-metrics-certs\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.141339 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.141312    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04432384-5c7b-4cf0-92eb-87d4beac19be-cert\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.141339 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.141329    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04432384-5c7b-4cf0-92eb-87d4beac19be-metrics-certs\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.216871 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.216810    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qpj9\" (UniqueName: \"kubernetes.io/projected/04432384-5c7b-4cf0-92eb-87d4beac19be-kube-api-access-6qpj9\") pod \"odh-model-controller-769875db56-2bsq2\" (UID: \"04432384-5c7b-4cf0-92eb-87d4beac19be\") " pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.241220 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.241198    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:40.380869 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.380831    5561 generic.go:356] "Generic (PLEG): container finished" podID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" containerID="8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a" exitCode=0
Oct 30 21:40:40.381036 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.380899    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"
Oct 30 21:40:40.381036 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.380905    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" event={"ID":"f6fa3f27-d6e7-4cab-95b0-330c75c088df","Type":"ContainerDied","Data":"8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a"}
Oct 30 21:40:40.381036 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.380944    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp" event={"ID":"f6fa3f27-d6e7-4cab-95b0-330c75c088df","Type":"ContainerDied","Data":"88e97655b66e6ac1e190e635a8d2fadef588e8747d6d2d39de2f3cfcd14ecc27"}
Oct 30 21:40:40.381036 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.380964    5561 scope.go:122] "RemoveContainer" containerID="8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a"
Oct 30 21:40:40.389054 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.389035    5561 scope.go:122] "RemoveContainer" containerID="8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a"
Oct 30 21:40:40.389317 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:40.389295    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a\": container with ID starting with 8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a not found: ID does not exist" containerID="8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a"
Oct 30 21:40:40.389370 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.389331    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a"} err="failed to get container status \"8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a\": rpc error: code = NotFound desc = could not find container \"8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a\": container with ID starting with 8b3467bc7131b9f82382e0cfd052c3dfec77459b958efe51d2f33fd50fcd638a not found: ID does not exist"
Oct 30 21:40:40.545616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.545544    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"]
Oct 30 21:40:40.551520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.551494    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp"]
Oct 30 21:40:40.655496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:40.655472    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-2bsq2"]
Oct 30 21:40:40.657595 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:40:40.657568    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04432384_5c7b_4cf0_92eb_87d4beac19be.slice/crio-2b47d2c89303fb6d973adf9930c98de8e3496301e7710d762ba2115751a6e96b WatchSource:0}: Error finding container 2b47d2c89303fb6d973adf9930c98de8e3496301e7710d762ba2115751a6e96b: Status 404 returned error can't find the container with id 2b47d2c89303fb6d973adf9930c98de8e3496301e7710d762ba2115751a6e96b
Oct 30 21:40:41.386157 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:41.386122    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-2bsq2" event={"ID":"04432384-5c7b-4cf0-92eb-87d4beac19be","Type":"ContainerStarted","Data":"2b47d2c89303fb6d973adf9930c98de8e3496301e7710d762ba2115751a6e96b"}
Oct 30 21:40:41.846004 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:41.845898    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6fa3f27-d6e7-4cab-95b0-330c75c088df" path="/var/lib/kubelet/pods/f6fa3f27-d6e7-4cab-95b0-330c75c088df/volumes"
Oct 30 21:40:44.401123 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:44.401088    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-2bsq2" event={"ID":"04432384-5c7b-4cf0-92eb-87d4beac19be","Type":"ContainerStarted","Data":"b844f0adeb2742cd83ce811cff2565c855711aba97d8f1db2053ab7e6a0442f1"}
Oct 30 21:40:44.401574 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:44.401233    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:44.520108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:44.520049    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-2bsq2" podStartSLOduration=2.410491002 podStartE2EDuration="5.520036843s" totalImagesPullingTime="3.109545841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:44.519983354 +0000 UTC m=+392.797676824" watchObservedRunningTime="2026-10-30 21:40:44.520036843 +0000 UTC m=+392.797730314"
Oct 30 21:40:49.380304 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:49.380275    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hrf8w"
Oct 30 21:40:52.980149 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:40:52.980111    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:40:54.095943 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.095910    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"]
Oct 30 21:40:54.097875 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.097859    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.204224 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.204188    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:40:54.204224 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.204172    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:40:54.204224 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.204202    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:40:54.204547 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.204529    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 21:40:54.204813 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.204798    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fqgfc\""
Oct 30 21:40:54.246202 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.246176    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"]
Oct 30 21:40:54.258292 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.258267    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-maas-api-tls\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.258419 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.258305    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm5km\" (UniqueName: \"kubernetes.io/projected/8e244428-d253-4412-88b5-4367cdb92209-kube-api-access-vm5km\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.258461 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.258449    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e244428-d253-4412-88b5-4367cdb92209-openshift-service-ca\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.258499 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.258479    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-metrics-certs\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.359325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.359297    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e244428-d253-4412-88b5-4367cdb92209-openshift-service-ca\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.359437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.359331    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-metrics-certs\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.359437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.359370    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-maas-api-tls\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.359437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.359401    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm5km\" (UniqueName: \"kubernetes.io/projected/8e244428-d253-4412-88b5-4367cdb92209-kube-api-access-vm5km\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.360047 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.360026    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e244428-d253-4412-88b5-4367cdb92209-openshift-service-ca\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.361851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.361831    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-metrics-certs\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.361934 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.361883    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-maas-api-tls\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.417047 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.417023    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm5km\" (UniqueName: \"kubernetes.io/projected/8e244428-d253-4412-88b5-4367cdb92209-kube-api-access-vm5km\") pod \"maas-api-6d669c7fcd-d8w5l\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:54.442637 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.442610    5561 generic.go:356] "Generic (PLEG): container finished" podID="04432384-5c7b-4cf0-92eb-87d4beac19be" containerID="b844f0adeb2742cd83ce811cff2565c855711aba97d8f1db2053ab7e6a0442f1" exitCode=1
Oct 30 21:40:54.442717 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.442641    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-2bsq2" event={"ID":"04432384-5c7b-4cf0-92eb-87d4beac19be","Type":"ContainerDied","Data":"b844f0adeb2742cd83ce811cff2565c855711aba97d8f1db2053ab7e6a0442f1"}
Oct 30 21:40:54.442951 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.442939    5561 scope.go:122] "RemoveContainer" containerID="b844f0adeb2742cd83ce811cff2565c855711aba97d8f1db2053ab7e6a0442f1"
Oct 30 21:40:54.708364 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:54.708326    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:55.081317 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:55.081140    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"]
Oct 30 21:40:55.084060 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:40:55.084028    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e244428_d253_4412_88b5_4367cdb92209.slice/crio-0ab501404b0a1b032ece1bd1c425eaaf87e6c33abde110799ccc3c654fbba116 WatchSource:0}: Error finding container 0ab501404b0a1b032ece1bd1c425eaaf87e6c33abde110799ccc3c654fbba116: Status 404 returned error can't find the container with id 0ab501404b0a1b032ece1bd1c425eaaf87e6c33abde110799ccc3c654fbba116
Oct 30 21:40:55.449098 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:55.449066    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-2bsq2" event={"ID":"04432384-5c7b-4cf0-92eb-87d4beac19be","Type":"ContainerStarted","Data":"ba0fcd467cc485685de0f5bb09584521aac863125460aeef909f2b836f8e3c65"}
Oct 30 21:40:55.449518 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:55.449279    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:40:55.450313 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:55.450288    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l" event={"ID":"8e244428-d253-4412-88b5-4367cdb92209","Type":"ContainerStarted","Data":"0ab501404b0a1b032ece1bd1c425eaaf87e6c33abde110799ccc3c654fbba116"}
Oct 30 21:40:57.460764 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:57.460719    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l" event={"ID":"8e244428-d253-4412-88b5-4367cdb92209","Type":"ContainerStarted","Data":"9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77"}
Oct 30 21:40:57.461161 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:57.460815    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:40:57.597518 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:57.597464    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l" podStartSLOduration=1.64132304 podStartE2EDuration="3.597450624s" totalImagesPullingTime="1.956127584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:57.590467258 +0000 UTC m=+405.868160729" watchObservedRunningTime="2026-10-30 21:40:57.597450624 +0000 UTC m=+405.875144094"
Oct 30 21:40:59.779459 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:40:59.779422    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:41:03.470610 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:03.470579    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:41:04.631532 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.631497    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7975887845-vbk67"]
Oct 30 21:41:04.643472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.643441    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.693580 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.693544    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7975887845-vbk67"]
Oct 30 21:41:04.736949 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.736916    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwbc\" (UniqueName: \"kubernetes.io/projected/a509d6f8-e623-45dc-a9d6-922a836a95f5-kube-api-access-gzwbc\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.737068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.736969    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-metrics-certs\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.737068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.737028    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a509d6f8-e623-45dc-a9d6-922a836a95f5-openshift-service-ca\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.737068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.737057    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-maas-api-tls\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.838099 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.838070    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwbc\" (UniqueName: \"kubernetes.io/projected/a509d6f8-e623-45dc-a9d6-922a836a95f5-kube-api-access-gzwbc\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.838266 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.838126    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-metrics-certs\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.838329 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.838306    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a509d6f8-e623-45dc-a9d6-922a836a95f5-openshift-service-ca\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.838406 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.838384    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-maas-api-tls\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.838964 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.838935    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a509d6f8-e623-45dc-a9d6-922a836a95f5-openshift-service-ca\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.840786 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.840755    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-metrics-certs\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.840920 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.840766    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-maas-api-tls\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.895240 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.895187    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwbc\" (UniqueName: \"kubernetes.io/projected/a509d6f8-e623-45dc-a9d6-922a836a95f5-kube-api-access-gzwbc\") pod \"maas-api-7975887845-vbk67\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") " pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:04.957952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:04.957927    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:05.386724 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:05.386701    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7975887845-vbk67"]
Oct 30 21:41:05.389189 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:41:05.389149    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda509d6f8_e623_45dc_a9d6_922a836a95f5.slice/crio-4ca3eb78ac30905907047e66ce5b20705aac66b641b72e5dbc48a7f4f95cd930 WatchSource:0}: Error finding container 4ca3eb78ac30905907047e66ce5b20705aac66b641b72e5dbc48a7f4f95cd930: Status 404 returned error can't find the container with id 4ca3eb78ac30905907047e66ce5b20705aac66b641b72e5dbc48a7f4f95cd930
Oct 30 21:41:05.496417 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:05.496391    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67" event={"ID":"a509d6f8-e623-45dc-a9d6-922a836a95f5","Type":"ContainerStarted","Data":"4ca3eb78ac30905907047e66ce5b20705aac66b641b72e5dbc48a7f4f95cd930"}
Oct 30 21:41:06.457471 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:06.457447    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-2bsq2"
Oct 30 21:41:06.502152 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:06.502122    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67" event={"ID":"a509d6f8-e623-45dc-a9d6-922a836a95f5","Type":"ContainerStarted","Data":"6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f"}
Oct 30 21:41:06.502245 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:06.502173    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:06.691374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:06.691292    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67" podStartSLOduration=1.7138839510000001 podStartE2EDuration="2.691277355s" totalImagesPullingTime="977.393404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:06.690653279 +0000 UTC m=+414.968346752" watchObservedRunningTime="2026-10-30 21:41:06.691277355 +0000 UTC m=+414.968970847"
Oct 30 21:41:12.512201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:12.512168    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 21:41:12.661335 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:12.661302    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"]
Oct 30 21:41:12.661593 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:12.661566    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l" podUID="8e244428-d253-4412-88b5-4367cdb92209" containerName="maas-api" containerID="cri-o://9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77" gracePeriod=30
Oct 30 21:41:12.896992 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:12.896969    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:41:13.004023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.003994    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e244428-d253-4412-88b5-4367cdb92209-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e244428-d253-4412-88b5-4367cdb92209-openshift-service-ca\") pod \"8e244428-d253-4412-88b5-4367cdb92209\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") "
Oct 30 21:41:13.004167 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.004040    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-maas-api-tls\") pod \"8e244428-d253-4412-88b5-4367cdb92209\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") "
Oct 30 21:41:13.004167 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.004067    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-metrics-certs\") pod \"8e244428-d253-4412-88b5-4367cdb92209\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") "
Oct 30 21:41:13.004167 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.004090    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e244428-d253-4412-88b5-4367cdb92209-kube-api-access-vm5km\" (UniqueName: \"kubernetes.io/projected/8e244428-d253-4412-88b5-4367cdb92209-kube-api-access-vm5km\") pod \"8e244428-d253-4412-88b5-4367cdb92209\" (UID: \"8e244428-d253-4412-88b5-4367cdb92209\") "
Oct 30 21:41:13.004440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.004410    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e244428-d253-4412-88b5-4367cdb92209-openshift-service-ca" pod "8e244428-d253-4412-88b5-4367cdb92209" (UID: "8e244428-d253-4412-88b5-4367cdb92209"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:13.006227 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.006199    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-metrics-certs" pod "8e244428-d253-4412-88b5-4367cdb92209" (UID: "8e244428-d253-4412-88b5-4367cdb92209"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:41:13.006227 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.006214    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-maas-api-tls" pod "8e244428-d253-4412-88b5-4367cdb92209" (UID: "8e244428-d253-4412-88b5-4367cdb92209"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:41:13.006400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.006320    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e244428-d253-4412-88b5-4367cdb92209-kube-api-access-vm5km" pod "8e244428-d253-4412-88b5-4367cdb92209" (UID: "8e244428-d253-4412-88b5-4367cdb92209"). InnerVolumeSpecName "kube-api-access-vm5km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:13.104676 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.104612    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e244428-d253-4412-88b5-4367cdb92209-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.104676 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.104639    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.104676 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.104648    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e244428-d253-4412-88b5-4367cdb92209-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.104676 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.104657    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm5km\" (UniqueName: \"kubernetes.io/projected/8e244428-d253-4412-88b5-4367cdb92209-kube-api-access-vm5km\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.534404 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.534366    5561 generic.go:356] "Generic (PLEG): container finished" podID="8e244428-d253-4412-88b5-4367cdb92209" containerID="9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77" exitCode=0
Oct 30 21:41:13.534825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.534418    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"
Oct 30 21:41:13.534825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.534420    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l" event={"ID":"8e244428-d253-4412-88b5-4367cdb92209","Type":"ContainerDied","Data":"9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77"}
Oct 30 21:41:13.534825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.534539    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l" event={"ID":"8e244428-d253-4412-88b5-4367cdb92209","Type":"ContainerDied","Data":"0ab501404b0a1b032ece1bd1c425eaaf87e6c33abde110799ccc3c654fbba116"}
Oct 30 21:41:13.534825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.534563    5561 scope.go:122] "RemoveContainer" containerID="9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77"
Oct 30 21:41:13.545133 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.545109    5561 scope.go:122] "RemoveContainer" containerID="9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77"
Oct 30 21:41:13.545400 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:41:13.545379    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77\": container with ID starting with 9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77 not found: ID does not exist" containerID="9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77"
Oct 30 21:41:13.545478 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.545409    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77"} err="failed to get container status \"9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77\": rpc error: code = NotFound desc = could not find container \"9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77\": container with ID starting with 9d239a0aefa152f7d352f8276b2f435141bbf0fccf178600593971e72fef9f77 not found: ID does not exist"
Oct 30 21:41:13.699245 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.699209    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"]
Oct 30 21:41:13.744748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.744710    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-d8w5l"]
Oct 30 21:41:13.843756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:41:13.843683    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e244428-d253-4412-88b5-4367cdb92209" path="/var/lib/kubelet/pods/8e244428-d253-4412-88b5-4367cdb92209/volumes"
Oct 30 21:41:22.979860 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:41:22.979820    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:41:52.979072 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:41:52.978988    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:42:22.979069 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:42:22.979027    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:42:40.188687 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.188655    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"]
Oct 30 21:42:40.194830 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.194801    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8e244428-d253-4412-88b5-4367cdb92209" containerName="maas-api"
Oct 30 21:42:40.194830 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.194825    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e244428-d253-4412-88b5-4367cdb92209" containerName="maas-api"
Oct 30 21:42:40.195015 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.194896    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e244428-d253-4412-88b5-4367cdb92209" containerName="maas-api"
Oct 30 21:42:40.197893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.197876    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.249736 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.249696    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"]
Oct 30 21:42:40.296950 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.296925    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.297073 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.296958    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.297073 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.296987    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bvnp\" (UniqueName: \"kubernetes.io/projected/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-kube-api-access-4bvnp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.297157 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.297128    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.297196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.297169    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.297196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.297186    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.298842 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.298824    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:42:40.298936 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.298889    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:42:40.299040 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.299027    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4xmws\""
Oct 30 21:42:40.300863 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.300846    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 21:42:40.397744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.397719    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.397837 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.397747    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.397837 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.397767    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bvnp\" (UniqueName: \"kubernetes.io/projected/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-kube-api-access-4bvnp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.397837 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.397809    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.397947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.397841    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.397947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.397857    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.398189 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.398164    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.398276 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.398239    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.398408 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.398389    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.399992 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.399965    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.400416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.400394    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.455115 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.455060    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bvnp\" (UniqueName: \"kubernetes.io/projected/a23bab1a-e64d-4c29-9e1b-941b7fdbdb34-kube-api-access-4bvnp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm\" (UID: \"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.509985 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.509962    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:40.877794 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.877570    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"]
Oct 30 21:42:40.879965 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:42:40.879937    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23bab1a_e64d_4c29_9e1b_941b7fdbdb34.slice/crio-7cd35661bfdea86d50c89a6b65c9e1257f8766bef59ed9530130cee42c4efa65 WatchSource:0}: Error finding container 7cd35661bfdea86d50c89a6b65c9e1257f8766bef59ed9530130cee42c4efa65: Status 404 returned error can't find the container with id 7cd35661bfdea86d50c89a6b65c9e1257f8766bef59ed9530130cee42c4efa65
Oct 30 21:42:40.883941 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.883923    5561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:42:40.889632 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:40.889608    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm" event={"ID":"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34","Type":"ContainerStarted","Data":"7cd35661bfdea86d50c89a6b65c9e1257f8766bef59ed9530130cee42c4efa65"}
Oct 30 21:42:43.742666 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:43.742632    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:42:43.903255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:43.903225    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm" event={"ID":"a23bab1a-e64d-4c29-9e1b-941b7fdbdb34","Type":"ContainerStarted","Data":"9b9c889ddf086676dce956401196694d1569a688fb6533712eb8ad682ab87282"}
Oct 30 21:42:44.022614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:44.022521    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm" podStartSLOduration=1.74225208 podStartE2EDuration="4.022506794s" totalImagesPullingTime="2.280254714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:44.021615356 +0000 UTC m=+512.299308826" watchObservedRunningTime="2026-10-30 21:42:44.022506794 +0000 UTC m=+512.300200297"
Oct 30 21:42:44.510607 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:44.510578    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:47.275189 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.275155    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"]
Oct 30 21:42:47.278714 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.278699    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.360562 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.360531    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.360723 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.360569    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2mnx\" (UniqueName: \"kubernetes.io/projected/1dff099d-a1aa-41e4-b2c3-3bd96da42601-kube-api-access-c2mnx\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.360723 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.360693    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.360828 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.360721    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.360828 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.360741    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dff099d-a1aa-41e4-b2c3-3bd96da42601-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.360910 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.360835    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.381917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.381892    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 21:42:47.383246 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.383227    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"]
Oct 30 21:42:47.461321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461297    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461329    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2mnx\" (UniqueName: \"kubernetes.io/projected/1dff099d-a1aa-41e4-b2c3-3bd96da42601-kube-api-access-c2mnx\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461410    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461429    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461445    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dff099d-a1aa-41e4-b2c3-3bd96da42601-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461482    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461797 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461776    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461913 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461883    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.461913 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.461898    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.463754 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.463734    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dff099d-a1aa-41e4-b2c3-3bd96da42601-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.464034 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.464013    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dff099d-a1aa-41e4-b2c3-3bd96da42601-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.518820 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.518792    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2mnx\" (UniqueName: \"kubernetes.io/projected/1dff099d-a1aa-41e4-b2c3-3bd96da42601-kube-api-access-c2mnx\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-9xc8j\" (UID: \"1dff099d-a1aa-41e4-b2c3-3bd96da42601\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.590673 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.590617    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:47.959652 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:47.959628    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"]
Oct 30 21:42:47.961962 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:42:47.961934    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dff099d_a1aa_41e4_b2c3_3bd96da42601.slice/crio-55a1693c18728b476df0e5b65c5399c1f7b4c994c361d0d0cd094ab059faf88e WatchSource:0}: Error finding container 55a1693c18728b476df0e5b65c5399c1f7b4c994c361d0d0cd094ab059faf88e: Status 404 returned error can't find the container with id 55a1693c18728b476df0e5b65c5399c1f7b4c994c361d0d0cd094ab059faf88e
Oct 30 21:42:48.930327 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:48.930257    5561 generic.go:356] "Generic (PLEG): container finished" podID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerID="3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4" exitCode=1
Oct 30 21:42:48.930719 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:48.930329    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-l7ph6" event={"ID":"e7d7cc94-b718-4145-a7a1-c7c91876d78e","Type":"ContainerDied","Data":"3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4"}
Oct 30 21:42:48.930719 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:48.930675    5561 scope.go:122] "RemoveContainer" containerID="3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4"
Oct 30 21:42:48.931978 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:48.931958    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j" event={"ID":"1dff099d-a1aa-41e4-b2c3-3bd96da42601","Type":"ContainerStarted","Data":"df3108aa306d6e5353de8353d44518accee948457258ebb16e463c60508c3772"}
Oct 30 21:42:48.932089 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:48.932000    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j" event={"ID":"1dff099d-a1aa-41e4-b2c3-3bd96da42601","Type":"ContainerStarted","Data":"55a1693c18728b476df0e5b65c5399c1f7b4c994c361d0d0cd094ab059faf88e"}
Oct 30 21:42:49.164636 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:49.164585    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j" podStartSLOduration=1.380907292 podStartE2EDuration="2.16456507s" totalImagesPullingTime="783.657778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:49.163479697 +0000 UTC m=+517.441173165" watchObservedRunningTime="2026-10-30 21:42:49.16456507 +0000 UTC m=+517.442258540"
Oct 30 21:42:49.591752 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:49.591721    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:49.937084 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:49.937055    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-l7ph6" event={"ID":"e7d7cc94-b718-4145-a7a1-c7c91876d78e","Type":"ContainerStarted","Data":"37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e"}
Oct 30 21:42:49.937437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:49.937396    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:42:50.510664 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:50.510642    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:50.523441 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:50.523421    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:50.952882 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:50.952853    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm"
Oct 30 21:42:51.905115 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:51.905081    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:42:52.978920 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:42:52.978881    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:42:57.591196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:57.591160    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:57.603614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:57.603589    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:42:57.980924 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:42:57.980901    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-9xc8j"
Oct 30 21:43:00.041076 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.041040    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"]
Oct 30 21:43:00.047113 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.047093    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.157921 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.157896    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 21:43:00.159371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.159339    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"]
Oct 30 21:43:00.161431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.161413    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2k6v\" (UniqueName: \"kubernetes.io/projected/1301ca09-349b-414d-bdf2-8297fa20c2cd-kube-api-access-h2k6v\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.161498 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.161441    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.161498 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.161464    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.161613 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.161598    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.161695 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.161681    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.161744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.161731    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1301ca09-349b-414d-bdf2-8297fa20c2cd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.262692 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.262662    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.262811 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.262697    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.262811 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.262733    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1301ca09-349b-414d-bdf2-8297fa20c2cd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.262811 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.262771    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2k6v\" (UniqueName: \"kubernetes.io/projected/1301ca09-349b-414d-bdf2-8297fa20c2cd-kube-api-access-h2k6v\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.262811 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.262791    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.263004 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.262821    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.263074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.263052    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.263210 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.263181    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.263210 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.263203    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.265014 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.264994    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1301ca09-349b-414d-bdf2-8297fa20c2cd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.265304 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.265282    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1301ca09-349b-414d-bdf2-8297fa20c2cd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.322335 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.322281    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2k6v\" (UniqueName: \"kubernetes.io/projected/1301ca09-349b-414d-bdf2-8297fa20c2cd-kube-api-access-h2k6v\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6\" (UID: \"1301ca09-349b-414d-bdf2-8297fa20c2cd\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.359168 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.359146    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:00.730053 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.730028    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"]
Oct 30 21:43:00.731633 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:43:00.731604    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1301ca09_349b_414d_bdf2_8297fa20c2cd.slice/crio-9262d8da7d41c0390c16d034789189c24f2fa51d9de30d536c73e9567445608c WatchSource:0}: Error finding container 9262d8da7d41c0390c16d034789189c24f2fa51d9de30d536c73e9567445608c: Status 404 returned error can't find the container with id 9262d8da7d41c0390c16d034789189c24f2fa51d9de30d536c73e9567445608c
Oct 30 21:43:00.943946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.943904    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 21:43:00.984636 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:00.984556    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" event={"ID":"1301ca09-349b-414d-bdf2-8297fa20c2cd","Type":"ContainerStarted","Data":"9262d8da7d41c0390c16d034789189c24f2fa51d9de30d536c73e9567445608c"}
Oct 30 21:43:01.989950 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:01.989912    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" event={"ID":"1301ca09-349b-414d-bdf2-8297fa20c2cd","Type":"ContainerStarted","Data":"9919b56f9d59cfe0f20c50587d27a44813a16ac14767bae09c20756038cc37e8"}
Oct 30 21:43:02.111215 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:02.111161    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podStartSLOduration=2.38168552 podStartE2EDuration="3.111145029s" totalImagesPullingTime="729.459509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:02.110478299 +0000 UTC m=+530.388171770" watchObservedRunningTime="2026-10-30 21:43:02.111145029 +0000 UTC m=+530.388838501"
Oct 30 21:43:02.359554 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:02.359477    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:03.542238 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.542208    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:43:03.742422 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.742393    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"]
Oct 30 21:43:03.746433 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.746414    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.802733 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.802678    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 21:43:03.854410 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.854387    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"]
Oct 30 21:43:03.893540 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.893505    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6lsg\" (UniqueName: \"kubernetes.io/projected/8bcf17cc-fbd2-471e-af39-384080e3d198-kube-api-access-j6lsg\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.893646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.893551    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.893646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.893591    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.893715 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.893671    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.893715 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.893694    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bcf17cc-fbd2-471e-af39-384080e3d198-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.893715 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.893713    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994141 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994118    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994160    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994191    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bcf17cc-fbd2-471e-af39-384080e3d198-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994217    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994395 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994267    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6lsg\" (UniqueName: \"kubernetes.io/projected/8bcf17cc-fbd2-471e-af39-384080e3d198-kube-api-access-j6lsg\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994395 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994300    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994600 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994578    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994738 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994710    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.994738 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.994647    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.996723 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.996703    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bcf17cc-fbd2-471e-af39-384080e3d198-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:03.996808 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:03.996760    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bcf17cc-fbd2-471e-af39-384080e3d198-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:04.058821 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:04.058767    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6lsg\" (UniqueName: \"kubernetes.io/projected/8bcf17cc-fbd2-471e-af39-384080e3d198-kube-api-access-j6lsg\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7\" (UID: \"8bcf17cc-fbd2-471e-af39-384080e3d198\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:04.359023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:04.358954    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:04.727298 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:04.727260    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"]
Oct 30 21:43:04.729588 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:43:04.729558    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bcf17cc_fbd2_471e_af39_384080e3d198.slice/crio-c80389c456dd6d02396202dfc8f5770cbcf1573e18c0323e4f794435a812f8b5 WatchSource:0}: Error finding container c80389c456dd6d02396202dfc8f5770cbcf1573e18c0323e4f794435a812f8b5: Status 404 returned error can't find the container with id c80389c456dd6d02396202dfc8f5770cbcf1573e18c0323e4f794435a812f8b5
Oct 30 21:43:05.004130 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:05.004064    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7" event={"ID":"8bcf17cc-fbd2-471e-af39-384080e3d198","Type":"ContainerStarted","Data":"c80389c456dd6d02396202dfc8f5770cbcf1573e18c0323e4f794435a812f8b5"}
Oct 30 21:43:06.009902 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.009867    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7" event={"ID":"8bcf17cc-fbd2-471e-af39-384080e3d198","Type":"ContainerStarted","Data":"8061cada6057c2027d7dcfb211f93c8bdeeb9fb7d6ff18bb3bbb9bd720421309"}
Oct 30 21:43:06.128610 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.128556    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7" podStartSLOduration=2.434836195 podStartE2EDuration="3.128543056s" totalImagesPullingTime="693.706861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:06.127523224 +0000 UTC m=+534.405216696" watchObservedRunningTime="2026-10-30 21:43:06.128543056 +0000 UTC m=+534.406236527"
Oct 30 21:43:06.359500 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.359424    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:06.803119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.803083    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"]
Oct 30 21:43:06.806604 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.806586    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:06.862253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.862220    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 21:43:06.913711 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.913682    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"]
Oct 30 21:43:06.920864 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.920844    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:06.920918 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.920877    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:06.920918 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.920908    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:06.921034 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.921019    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgggx\" (UniqueName: \"kubernetes.io/projected/ee336b48-8449-45ec-a82d-7931efe627ee-kube-api-access-wgggx\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:06.921093 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.921080    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee336b48-8449-45ec-a82d-7931efe627ee-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:06.921130 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:06.921117    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.021822 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.021791    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.021829    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.021856    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.021971    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgggx\" (UniqueName: \"kubernetes.io/projected/ee336b48-8449-45ec-a82d-7931efe627ee-kube-api-access-wgggx\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.022008    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee336b48-8449-45ec-a82d-7931efe627ee-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.022152    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.022152    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022522 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.022265    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.022522 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.022407    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.024038 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.024021    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee336b48-8449-45ec-a82d-7931efe627ee-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.024276 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.024260    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee336b48-8449-45ec-a82d-7931efe627ee-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.082460 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.082408    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgggx\" (UniqueName: \"kubernetes.io/projected/ee336b48-8449-45ec-a82d-7931efe627ee-kube-api-access-wgggx\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-z2gfw\" (UID: \"ee336b48-8449-45ec-a82d-7931efe627ee\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.119235 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.119213    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:07.490412 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:07.490385    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"]
Oct 30 21:43:07.492046 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:43:07.492021    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee336b48_8449_45ec_a82d_7931efe627ee.slice/crio-5c9caf7e338cfaf736fe3afa1e679be01cddd0311c77c5093df8c18fa540e5f7 WatchSource:0}: Error finding container 5c9caf7e338cfaf736fe3afa1e679be01cddd0311c77c5093df8c18fa540e5f7: Status 404 returned error can't find the container with id 5c9caf7e338cfaf736fe3afa1e679be01cddd0311c77c5093df8c18fa540e5f7
Oct 30 21:43:08.019715 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:08.019682    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw" event={"ID":"ee336b48-8449-45ec-a82d-7931efe627ee","Type":"ContainerStarted","Data":"5c9caf7e338cfaf736fe3afa1e679be01cddd0311c77c5093df8c18fa540e5f7"}
Oct 30 21:43:09.026397 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:09.026344    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw" event={"ID":"ee336b48-8449-45ec-a82d-7931efe627ee","Type":"ContainerStarted","Data":"cef252c06da629e24e4e7814abf31b2151f75b2be555dcb466386ee5267c4103"}
Oct 30 21:43:09.119941 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:09.119908    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:09.161617 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:09.161569    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw" podStartSLOduration=2.416717582 podStartE2EDuration="3.161556932s" totalImagesPullingTime="744.83935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:09.159798323 +0000 UTC m=+537.437491795" watchObservedRunningTime="2026-10-30 21:43:09.161556932 +0000 UTC m=+537.439250401"
Oct 30 21:43:10.359369 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.359325    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:10.372249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.372224    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:10.834806 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.834775    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"]
Oct 30 21:43:10.838387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.838371    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852001    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852129 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852057    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852174 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852139    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852174 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852166    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzfzc\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-kube-api-access-fzfzc\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852197    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852245    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852267    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6de057-3fef-4a72-b7f3-e685586bd27d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852337 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852289    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.852407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.852394    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.953071 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953039    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.953317 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953280    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.953444 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953320    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzfzc\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-kube-api-access-fzfzc\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.953573 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953559    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.953897 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953874    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.954029 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953926    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.954029 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953959    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.954029 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.953985    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6de057-3fef-4a72-b7f3-e685586bd27d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.954029 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.954021    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.954287 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.954050    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.954287 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.954091    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.954756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.954734    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.955172 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.955151    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.955899 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.955876    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"]
Oct 30 21:43:10.956726 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.956707    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xn2np\""
Oct 30 21:43:10.956829 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.956751    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.956906 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.956882    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6de057-3fef-4a72-b7f3-e685586bd27d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:10.957609 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:10.957584    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:11.020560 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:11.020535    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:11.020755 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:11.020736    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzfzc\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-kube-api-access-fzfzc\") pod \"maas-default-gateway-openshift-default-74854d8ccf-jhtzt\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:11.045340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:11.045320    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:43:11.151055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:11.151032    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:11.590342 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:11.590311    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"]
Oct 30 21:43:11.592951 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:43:11.592922    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6de057_3fef_4a72_b7f3_e685586bd27d.slice/crio-e27e9f87893b24168838b397b67903c1f17ba6c6387228ca49b0809956ab2d87 WatchSource:0}: Error finding container e27e9f87893b24168838b397b67903c1f17ba6c6387228ca49b0809956ab2d87: Status 404 returned error can't find the container with id e27e9f87893b24168838b397b67903c1f17ba6c6387228ca49b0809956ab2d87
Oct 30 21:43:12.039619 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:12.039589    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" event={"ID":"df6de057-3fef-4a72-b7f3-e685586bd27d","Type":"ContainerStarted","Data":"e27e9f87893b24168838b397b67903c1f17ba6c6387228ca49b0809956ab2d87"}
Oct 30 21:43:14.360102 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:14.360064    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:14.374741 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:14.374718    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:15.067557 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:15.067525    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7"
Oct 30 21:43:15.595265 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:15.595232    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:15.595574 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:15.595299    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:15.595574 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:15.595330    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:16.074067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:16.074033    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" event={"ID":"df6de057-3fef-4a72-b7f3-e685586bd27d","Type":"ContainerStarted","Data":"f025585e80d75292ea27f51a8fec4512c854a5ca76316a7d6e11e7e92b5672f2"}
Oct 30 21:43:16.164375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:16.160168    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:16.164375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:16.160327    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:43:16.164375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:16.160400    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 21:43:17.120171 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:17.120140    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:17.132363 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:17.132332    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:17.165041 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:17.164652    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:43:17.165041 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:17.164702    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 21:43:17.263372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:17.261999    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podStartSLOduration=3.262497061 podStartE2EDuration="7.261980061s" totalImagesPullingTime="3.999483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:16.215198086 +0000 UTC m=+544.492891557" watchObservedRunningTime="2026-10-30 21:43:17.261980061 +0000 UTC m=+545.539673534"
Oct 30 21:43:18.094816 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:18.094789    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-z2gfw"
Oct 30 21:43:18.157654 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:18.157622    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:43:18.158185 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:18.158157    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 21:43:19.160674 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:19.160640    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:43:19.161242 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:19.161209    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 21:43:20.168377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:20.166324    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:43:20.168377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:20.166409    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 21:43:21.166375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:21.165648    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:43:21.166375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:21.165706    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 21:43:21.989030 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:21.988953    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:43:22.151786 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:22.151755    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:43:22.151938 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:22.151806    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 21:43:22.980627 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:43:22.980593    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:43:23.801528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:23.801496    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:23.801971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:23.801824    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:23.802720 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:23.802698    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:43:25.878650 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.878621    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"]
Oct 30 21:43:25.881732 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.881715    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.983500 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.983470    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"]
Oct 30 21:43:25.990755 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.990726    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.990877 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.990855    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.990926 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.990892    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.990926 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.990923    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.991008 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.990946    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.991044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.990996    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z7sw\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-kube-api-access-9z7sw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.991083 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.991072    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.991119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.991113    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:25.991159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:25.991129    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092390 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092362    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092400    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092429    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092448    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092470    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7sw\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-kube-api-access-9z7sw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092519    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092824 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092569    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092824 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092601    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092824 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092626    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.092824 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.092811    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.093232 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.093210    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.093324 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.093218    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.093324 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.093276    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.093438 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.093327    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.094740 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.094720    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.095364 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.095337    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.152243 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.152183    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.152382 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.152361    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z7sw\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-kube-api-access-9z7sw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-p8khq\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.193421 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.193397    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:26.626081 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.626053    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"]
Oct 30 21:43:26.627558 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:43:26.627531    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef00f19_fa1a_4a53_af23_534b63ed24a2.slice/crio-cc6e16e4abbd438c831643b1d52a407e6883b73d56e22b3e1ba41b46e13b1a19 WatchSource:0}: Error finding container cc6e16e4abbd438c831643b1d52a407e6883b73d56e22b3e1ba41b46e13b1a19: Status 404 returned error can't find the container with id cc6e16e4abbd438c831643b1d52a407e6883b73d56e22b3e1ba41b46e13b1a19
Oct 30 21:43:26.630712 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.630678    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:26.630793 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.630745    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:26.630793 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:26.630773    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:27.127467 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:27.127430    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" event={"ID":"8ef00f19-fa1a-4a53-af23-534b63ed24a2","Type":"ContainerStarted","Data":"93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc"}
Oct 30 21:43:27.127841 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:27.127484    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" event={"ID":"8ef00f19-fa1a-4a53-af23-534b63ed24a2","Type":"ContainerStarted","Data":"cc6e16e4abbd438c831643b1d52a407e6883b73d56e22b3e1ba41b46e13b1a19"}
Oct 30 21:43:27.194512 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:27.194477    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:27.195958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:27.195933    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:43:27.196100 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:27.195985    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 21:43:28.194336 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:28.194304    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:43:28.194722 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:28.194379    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 21:43:29.194728 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:29.194693    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:43:29.195117 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:29.194747    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 21:43:30.193945 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:30.193904    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:43:30.194137 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:30.193962    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 21:43:31.193757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:31.193721    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:43:31.194135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:31.193784    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 21:43:32.194645 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:32.194613    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:43:32.195041 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:32.194671    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 21:43:33.194607 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:33.194575    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:43:33.194805 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:33.194628    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 21:43:34.783315 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:34.783286    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:34.783785 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:34.783688    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:34.784489 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:34.784468    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:43:34.968883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:43:34.968835    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podStartSLOduration=9.968823858 podStartE2EDuration="9.968823858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:27.26892488 +0000 UTC m=+555.546618356" watchObservedRunningTime="2026-10-30 21:43:34.968823858 +0000 UTC m=+563.246517329"
Oct 30 21:43:52.978938 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:43:52.978900    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:44:11.905182 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:11.905155    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:44:11.906344 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:11.906325    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:44:22.981007 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:44:22.980968    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:44:26.393821 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.393788    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"]
Oct 30 21:44:26.396965 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.396948    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.528907 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.528877    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.528932    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.529053    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529191 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.529082    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de0bd7-f0de-46c2-94cb-d18f6836da70-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529191 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.529169    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529301 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.529212    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529301 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.529259    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnb7z\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-kube-api-access-dnb7z\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.529346    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.529416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.529402    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.602011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.601983    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"]
Oct 30 21:44:26.630578 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630552    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.630686 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630581    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de0bd7-f0de-46c2-94cb-d18f6836da70-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.630686 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630609    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.630686 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630633    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.630686 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630660    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnb7z\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-kube-api-access-dnb7z\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.630884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630689    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.630884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630717    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.630884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630808    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.631023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630976    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.631023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.630989    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.631145 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.631112    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.631289 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.631269    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.631371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.631319    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.631457 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.631437    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de0bd7-f0de-46c2-94cb-d18f6836da70-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.632888 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.632870    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.633136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.633119    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.688653 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.688605    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.689036 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.689019    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnb7z\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-kube-api-access-dnb7z\") pod \"maas-default-gateway-openshift-default-74854d8ccf-qxxgq\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:26.709590 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:26.709573    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:27.161933 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.161906    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"]
Oct 30 21:44:27.163956 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:44:27.163931    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93de0bd7_f0de_46c2_94cb_d18f6836da70.slice/crio-da54fdf1b0e59921904794b31b2a49072329907e12b103ed8a780f8dd7470782 WatchSource:0}: Error finding container da54fdf1b0e59921904794b31b2a49072329907e12b103ed8a780f8dd7470782: Status 404 returned error can't find the container with id da54fdf1b0e59921904794b31b2a49072329907e12b103ed8a780f8dd7470782
Oct 30 21:44:27.166004 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.165971    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:27.166085 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.166039    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:27.166085 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.166074    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:27.422541 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.422468    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" event={"ID":"93de0bd7-f0de-46c2-94cb-d18f6836da70","Type":"ContainerStarted","Data":"f22a8b47ca235f2918a4ecdc3db117d2fd7847e1a02f232a6b7f1911e823ee6b"}
Oct 30 21:44:27.422541 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.422521    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" event={"ID":"93de0bd7-f0de-46c2-94cb-d18f6836da70","Type":"ContainerStarted","Data":"da54fdf1b0e59921904794b31b2a49072329907e12b103ed8a780f8dd7470782"}
Oct 30 21:44:27.545952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.545892    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podStartSLOduration=1.545875517 podStartE2EDuration="1.545875517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:27.545758912 +0000 UTC m=+615.823452384" watchObservedRunningTime="2026-10-30 21:44:27.545875517 +0000 UTC m=+615.823568989"
Oct 30 21:44:27.710005 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:27.709927    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:28.711273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:28.711230    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:44:28.711796 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:28.711297    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:44:29.710248 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:29.710217    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:44:29.710444 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:29.710272    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:44:30.710943 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:30.710907    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:44:30.711325 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:30.710961    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:44:31.710778 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:31.710745    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:44:31.710956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:31.710796    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:44:32.710294 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:32.710262    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:44:32.710699 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:32.710314    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:44:33.710883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:33.710847    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:44:33.711250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:33.710903    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:44:35.337203 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:35.337168    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:35.337629 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:35.337558    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:35.338302 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:44:35.338275    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:44:52.978310 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:44:52.978221    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:45:22.980327 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:45:22.980287    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:45:39.837340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.837306    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"]
Oct 30 21:45:39.840908 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.840888    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.863798 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.863774    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.863926 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.863805    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmjdc\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-kube-api-access-qmjdc\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.863926 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.863840    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.864068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.864046    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad9b344-5973-422c-ae1b-43df366ee690-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.864120 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.864085    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.864181 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.864159    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.864222 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.864197    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.864311 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.864294    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad9b344-5973-422c-ae1b-43df366ee690-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.864394 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.864336    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.893937 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.893912    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c79cb5-openshift-default-dockercfg-7289d\""
Oct 30 21:45:39.946860 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.946828    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"]
Oct 30 21:45:39.965495 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965469    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965611 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965501    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmjdc\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-kube-api-access-qmjdc\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965611 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965520    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965611 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965574    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad9b344-5973-422c-ae1b-43df366ee690-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965611 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965590    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965611 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965607    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965622    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965666    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad9b344-5973-422c-ae1b-43df366ee690-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965699    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.965893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965845    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.966052 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.965916    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.966052 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.966006    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.966366 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.966333    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.966547 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.966523    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad9b344-5973-422c-ae1b-43df366ee690-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.967836 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.967817    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:39.968330 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:39.968310    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad9b344-5973-422c-ae1b-43df366ee690-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:40.025896 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.025851    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmjdc\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-kube-api-access-qmjdc\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:40.027970 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.027941    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:40.145198 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.145166    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"]
Oct 30 21:45:40.149764 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.149747    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.154646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.154625    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:40.205250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.204458    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-80c366-openshift-default-dockercfg-jqtkq\""
Oct 30 21:45:40.211889 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.211848    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"]
Oct 30 21:45:40.268969 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.268935    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269098 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.268974    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2mvz\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-kube-api-access-x2mvz\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269098 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.269080    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/498354f6-78ca-4228-b51b-f42ad382b63b-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.269124    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.269158    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269255 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.269236    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269292 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.269264    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/498354f6-78ca-4228-b51b-f42ad382b63b-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269393 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.269325    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.269458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.269396    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370489 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370457    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/498354f6-78ca-4228-b51b-f42ad382b63b-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370671 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370513    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370671 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370550    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370671 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370569    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370671 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370589    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/498354f6-78ca-4228-b51b-f42ad382b63b-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370671 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370634    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370951 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370720    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370951 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370759    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.370951 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.370792    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2mvz\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-kube-api-access-x2mvz\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.371107 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.371072    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.371440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.371390    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.371440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.371416    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/498354f6-78ca-4228-b51b-f42ad382b63b-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.371440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.371432    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.371620 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.371494    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.373033 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.373011    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.373267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.373249    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/498354f6-78ca-4228-b51b-f42ad382b63b-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.429023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.428960    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.429391 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.429345    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2mvz\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-kube-api-access-x2mvz\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.464455 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.464431    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:40.601516 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.601487    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"]
Oct 30 21:45:40.603564 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:45:40.603535    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad9b344_5973_422c_ae1b_43df366ee690.slice/crio-1c9a98c84ce7d18c2b35efe2eeef7eca34a558f72654dbea7ed33e6367bd9859 WatchSource:0}: Error finding container 1c9a98c84ce7d18c2b35efe2eeef7eca34a558f72654dbea7ed33e6367bd9859: Status 404 returned error can't find the container with id 1c9a98c84ce7d18c2b35efe2eeef7eca34a558f72654dbea7ed33e6367bd9859
Oct 30 21:45:40.605699 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.605668    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:40.605788 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.605736    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:40.605788 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.605779    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:40.758871 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.758788    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" event={"ID":"6ad9b344-5973-422c-ae1b-43df366ee690","Type":"ContainerStarted","Data":"df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5"}
Oct 30 21:45:40.758871 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.758850    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" event={"ID":"6ad9b344-5973-422c-ae1b-43df366ee690","Type":"ContainerStarted","Data":"1c9a98c84ce7d18c2b35efe2eeef7eca34a558f72654dbea7ed33e6367bd9859"}
Oct 30 21:45:40.842829 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.842793    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"]
Oct 30 21:45:40.851380 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:45:40.850441    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod498354f6_78ca_4228_b51b_f42ad382b63b.slice/crio-c95e86126825508decbc2ea635df3aa9f09beac6d702c74a52e8fdb5fe12b805 WatchSource:0}: Error finding container c95e86126825508decbc2ea635df3aa9f09beac6d702c74a52e8fdb5fe12b805: Status 404 returned error can't find the container with id c95e86126825508decbc2ea635df3aa9f09beac6d702c74a52e8fdb5fe12b805
Oct 30 21:45:40.852854 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.852820    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:40.852961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.852886    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:40.852961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.852916    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:40.881628 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:40.881575    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podStartSLOduration=1.881557457 podStartE2EDuration="1.881557457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:40.879846253 +0000 UTC m=+689.157539724" watchObservedRunningTime="2026-10-30 21:45:40.881557457 +0000 UTC m=+689.159250931"
Oct 30 21:45:41.097601 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.097513    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"]
Oct 30 21:45:41.101477 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.101455    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.155333 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.155299    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:41.157873 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.157845    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-dda9f5-openshift-default-dockercfg-qx2fq\""
Oct 30 21:45:41.159942 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.159912    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"]
Oct 30 21:45:41.161427 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.161404    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:41.179033 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179001    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c0dffe-c267-4356-aa24-03b78c24dd49-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179131 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179054    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179199 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179180    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179221    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179295 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179252    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qfgm\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-kube-api-access-4qfgm\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179395 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179375    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179464 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179438    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179506 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179494    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.179603 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.179586    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.280986 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.280952    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c0dffe-c267-4356-aa24-03b78c24dd49-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.280995    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281036    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281055    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281074    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qfgm\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-kube-api-access-4qfgm\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281093    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281532 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281511    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281663 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281316    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281723    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281747 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281648    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281900 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281786    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c0dffe-c267-4356-aa24-03b78c24dd49-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.281900 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281819    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.282016 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.281959    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.282142 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.282120    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.283618 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.283600    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.283996 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.283978    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.366185 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.366093    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.367310 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.367277    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qfgm\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-kube-api-access-4qfgm\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.414530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.414492    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:41.764909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.764868    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" event={"ID":"498354f6-78ca-4228-b51b-f42ad382b63b","Type":"ContainerStarted","Data":"010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d"}
Oct 30 21:45:41.765085 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.764928    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" event={"ID":"498354f6-78ca-4228-b51b-f42ad382b63b","Type":"ContainerStarted","Data":"c95e86126825508decbc2ea635df3aa9f09beac6d702c74a52e8fdb5fe12b805"}
Oct 30 21:45:41.765175 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.765144    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:41.765911 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.765895    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:45:41.789401 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.789383    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"]
Oct 30 21:45:41.791420 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:45:41.791397    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c0dffe_c267_4356_aa24_03b78c24dd49.slice/crio-cab4e85ca09b8696de504b47beaea50c861db83a9fcc855976108c9841959637 WatchSource:0}: Error finding container cab4e85ca09b8696de504b47beaea50c861db83a9fcc855976108c9841959637: Status 404 returned error can't find the container with id cab4e85ca09b8696de504b47beaea50c861db83a9fcc855976108c9841959637
Oct 30 21:45:41.793519 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.793490    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:41.793594 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.793552    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:41.793594 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.793582    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:41.886700 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:41.886639    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podStartSLOduration=1.886620763 podStartE2EDuration="1.886620763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:41.885960954 +0000 UTC m=+690.163654427" watchObservedRunningTime="2026-10-30 21:45:41.886620763 +0000 UTC m=+690.164314234"
Oct 30 21:45:42.472367 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:42.468755    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:42.486036 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:42.486011    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:42.772009 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:42.771925    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" event={"ID":"e1c0dffe-c267-4356-aa24-03b78c24dd49","Type":"ContainerStarted","Data":"b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4"}
Oct 30 21:45:42.772009 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:42.771979    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" event={"ID":"e1c0dffe-c267-4356-aa24-03b78c24dd49","Type":"ContainerStarted","Data":"cab4e85ca09b8696de504b47beaea50c861db83a9fcc855976108c9841959637"}
Oct 30 21:45:42.772387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:42.772346    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:42.773212 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:42.773195    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:45:42.893938 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:42.893896    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" podStartSLOduration=1.893884629 podStartE2EDuration="1.893884629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:42.892522617 +0000 UTC m=+691.170216092" watchObservedRunningTime="2026-10-30 21:45:42.893884629 +0000 UTC m=+691.171578100"
Oct 30 21:45:43.414797 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:43.414769    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:43.419266 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:43.419243    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:43.777588 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:43.777497    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:43.778119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:43.778101    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:45:50.315655 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.315622    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"]
Oct 30 21:45:50.319431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.319412    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.372598 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.372566    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-557708-serving-cert\""
Oct 30 21:45:50.409612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.409552    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rt29\" (UniqueName: \"kubernetes.io/projected/2aac0e94-fd07-4e94-8c32-959d287f689c-kube-api-access-2rt29\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.409804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.409617    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-metrics-certs\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.409804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.409680    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.409804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.409769    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aac0e94-fd07-4e94-8c32-959d287f689c-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.425085 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.425054    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"]
Oct 30 21:45:50.510927 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.510895    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aac0e94-fd07-4e94-8c32-959d287f689c-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.511091 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.511003    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rt29\" (UniqueName: \"kubernetes.io/projected/2aac0e94-fd07-4e94-8c32-959d287f689c-kube-api-access-2rt29\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.511091 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.511026    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-metrics-certs\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.511205 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.511177    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.511379 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:45:50.511361    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-serving-cert: secret "maas-api-e2e-worker-w6-557708-serving-cert" not found
Oct 30 21:45:50.511484 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:45:50.511429    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls podName:2aac0e94-fd07-4e94-8c32-959d287f689c nodeName:}" failed. No retries permitted until 2026-10-30 21:45:51.01140978 +0000 UTC m=+699.289103252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls") pod "maas-api-e2e-worker-w6-557708-6f4d568455-grnqb" (UID: "2aac0e94-fd07-4e94-8c32-959d287f689c") : secret "maas-api-e2e-worker-w6-557708-serving-cert" not found
Oct 30 21:45:50.511792 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.511776    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aac0e94-fd07-4e94-8c32-959d287f689c-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.513440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.513418    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-metrics-certs\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:50.596680 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:50.596621    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rt29\" (UniqueName: \"kubernetes.io/projected/2aac0e94-fd07-4e94-8c32-959d287f689c-kube-api-access-2rt29\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:51.026304 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:51.026269    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:51.028630 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:51.028610    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls\") pod \"maas-api-e2e-worker-w6-557708-6f4d568455-grnqb\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:51.231931 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:51.231899    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:51.601326 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:51.601303    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"]
Oct 30 21:45:51.603521 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:45:51.603493    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aac0e94_fd07_4e94_8c32_959d287f689c.slice/crio-88223187ea90e0173d61b6db0f3b18889d8f921c7b5b62280b65ba0b6937dc87 WatchSource:0}: Error finding container 88223187ea90e0173d61b6db0f3b18889d8f921c7b5b62280b65ba0b6937dc87: Status 404 returned error can't find the container with id 88223187ea90e0173d61b6db0f3b18889d8f921c7b5b62280b65ba0b6937dc87
Oct 30 21:45:51.813802 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:51.813770    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb" event={"ID":"2aac0e94-fd07-4e94-8c32-959d287f689c","Type":"ContainerStarted","Data":"88223187ea90e0173d61b6db0f3b18889d8f921c7b5b62280b65ba0b6937dc87"}
Oct 30 21:45:52.819710 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:52.819626    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb" event={"ID":"2aac0e94-fd07-4e94-8c32-959d287f689c","Type":"ContainerStarted","Data":"ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04"}
Oct 30 21:45:52.820063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:52.819804    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:52.938215 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:52.938160    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb" podStartSLOduration=2.095722669 podStartE2EDuration="2.938146503s" totalImagesPullingTime="842.423834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:52.937719416 +0000 UTC m=+701.215412887" watchObservedRunningTime="2026-10-30 21:45:52.938146503 +0000 UTC m=+701.215839973"
Oct 30 21:45:52.979758 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:45:52.979725    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:45:55.723305 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.723270    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"]
Oct 30 21:45:55.738258 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.738213    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.781415 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.781387    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wpbj\" (UniqueName: \"kubernetes.io/projected/ac67565e-5766-4d7a-9299-ce582e6e320f-kube-api-access-9wpbj\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.781568 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.781514    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-metrics-certs\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.781568 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.781546    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac67565e-5766-4d7a-9299-ce582e6e320f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.781646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.781599    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.791430 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.791406    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-80c366-serving-cert\""
Oct 30 21:45:55.794021 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.793993    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"]
Oct 30 21:45:55.883058 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.883029    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wpbj\" (UniqueName: \"kubernetes.io/projected/ac67565e-5766-4d7a-9299-ce582e6e320f-kube-api-access-9wpbj\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.883241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.883115    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-metrics-certs\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.883241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.883145    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac67565e-5766-4d7a-9299-ce582e6e320f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.883241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.883172    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.883428 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:45:55.883296    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-serving-cert: secret "maas-api-e2e-worker-w1-80c366-serving-cert" not found
Oct 30 21:45:55.883428 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:45:55.883380    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls podName:ac67565e-5766-4d7a-9299-ce582e6e320f nodeName:}" failed. No retries permitted until 2026-10-30 21:45:56.383336108 +0000 UTC m=+704.661029571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls") pod "maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q" (UID: "ac67565e-5766-4d7a-9299-ce582e6e320f") : secret "maas-api-e2e-worker-w1-80c366-serving-cert" not found
Oct 30 21:45:55.884100 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.884076    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac67565e-5766-4d7a-9299-ce582e6e320f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.885674 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.885653    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-metrics-certs\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:55.941615 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:55.941584    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wpbj\" (UniqueName: \"kubernetes.io/projected/ac67565e-5766-4d7a-9299-ce582e6e320f-kube-api-access-9wpbj\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:56.386884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:56.386847    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:56.389176 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:56.389146    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:56.656875 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:56.656800    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:57.027925 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:57.027903    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"]
Oct 30 21:45:57.029553 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:45:57.029525    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac67565e_5766_4d7a_9299_ce582e6e320f.slice/crio-521c6f70f767cf3247a86b1c808f2f4fe1d071da7d372d228806ac48deec177f WatchSource:0}: Error finding container 521c6f70f767cf3247a86b1c808f2f4fe1d071da7d372d228806ac48deec177f: Status 404 returned error can't find the container with id 521c6f70f767cf3247a86b1c808f2f4fe1d071da7d372d228806ac48deec177f
Oct 30 21:45:57.844027 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:57.844001    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q" event={"ID":"ac67565e-5766-4d7a-9299-ce582e6e320f","Type":"ContainerStarted","Data":"521c6f70f767cf3247a86b1c808f2f4fe1d071da7d372d228806ac48deec177f"}
Oct 30 21:45:58.543817 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.543782    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"]
Oct 30 21:45:58.547364 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.547336    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:58.605084 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.605044    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"]
Oct 30 21:45:58.616921 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.616895    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bphsx\" (UniqueName: \"kubernetes.io/projected/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-kube-api-access-bphsx\") pod \"payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:58.617072 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.616997    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:58.678449 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.678418    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-9b51e1-dockercfg-mqg58\""
Oct 30 21:45:58.678932 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.678785    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-9b51e1\""
Oct 30 21:45:58.717682 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.717653    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bphsx\" (UniqueName: \"kubernetes.io/projected/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-kube-api-access-bphsx\") pod \"payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:58.717816 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.717717    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:58.718395 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.718377    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:58.777282 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.777252    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bphsx\" (UniqueName: \"kubernetes.io/projected/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-kube-api-access-bphsx\") pod \"payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:58.829273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.829216    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 21:45:58.849992 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.849961    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q" event={"ID":"ac67565e-5766-4d7a-9299-ce582e6e320f","Type":"ContainerStarted","Data":"0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04"}
Oct 30 21:45:58.850111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.850038    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:45:58.856562 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.856539    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:45:58.859934 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:58.859913    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:45:59.065416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:59.065332    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q" podStartSLOduration=3.286973693 podStartE2EDuration="4.065314944s" totalImagesPullingTime="778.341251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:59.063111967 +0000 UTC m=+707.340805440" watchObservedRunningTime="2026-10-30 21:45:59.065314944 +0000 UTC m=+707.343008416"
Oct 30 21:45:59.237160 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:59.237134    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"]
Oct 30 21:45:59.239628 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:45:59.239579    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b01064_b927_4e4b_9cc9_c5d7dd1d6738.slice/crio-c4f3525ac04959115034494e57dd5c1b622bea64c2a6914e8f6b65beb1a06fbd WatchSource:0}: Error finding container c4f3525ac04959115034494e57dd5c1b622bea64c2a6914e8f6b65beb1a06fbd: Status 404 returned error can't find the container with id c4f3525ac04959115034494e57dd5c1b622bea64c2a6914e8f6b65beb1a06fbd
Oct 30 21:45:59.856640 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:45:59.856604    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" event={"ID":"99b01064-b927-4e4b-9cc9-c5d7dd1d6738","Type":"ContainerStarted","Data":"c4f3525ac04959115034494e57dd5c1b622bea64c2a6914e8f6b65beb1a06fbd"}
Oct 30 21:46:01.097749 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.097713    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"]
Oct 30 21:46:01.104406 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.104382    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.157006 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.156967    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.184064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.184025    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm5gw\" (UniqueName: \"kubernetes.io/projected/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-kube-api-access-hm5gw\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.186748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.186715    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-metrics-certs\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.190250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.187126    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.234303 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.234274    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-c79cb5-serving-cert\""
Oct 30 21:46:01.236924 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.236889    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"]
Oct 30 21:46:01.288011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.287969    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.288173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.288080    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.288173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.288107    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm5gw\" (UniqueName: \"kubernetes.io/projected/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-kube-api-access-hm5gw\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.288253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.288199    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-metrics-certs\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.288955 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:01.288929    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-serving-cert: secret "maas-api-e2e-worker-w2-c79cb5-serving-cert" not found
Oct 30 21:46:01.289095 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:01.289014    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls podName:a4ca4d2e-b84a-42d0-8636-cfc3c3a55935 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:01.788994332 +0000 UTC m=+710.066687782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls") pod "maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j" (UID: "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935") : secret "maas-api-e2e-worker-w2-c79cb5-serving-cert" not found
Oct 30 21:46:01.296396 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.292931    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.296396 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.294068    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-metrics-certs\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.350205 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.350138    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm5gw\" (UniqueName: \"kubernetes.io/projected/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-kube-api-access-hm5gw\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.803723 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.803683    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:01.806470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:01.806443    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:02.023484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:02.023451    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:02.400139 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:02.400116    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"]
Oct 30 21:46:02.401632 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:46:02.401610    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ca4d2e_b84a_42d0_8636_cfc3c3a55935.slice/crio-4dcda33c68806c315f30d412c3dcc3e1a0f55bad09d922f2fc1c85fc89356b2c WatchSource:0}: Error finding container 4dcda33c68806c315f30d412c3dcc3e1a0f55bad09d922f2fc1c85fc89356b2c: Status 404 returned error can't find the container with id 4dcda33c68806c315f30d412c3dcc3e1a0f55bad09d922f2fc1c85fc89356b2c
Oct 30 21:46:02.873506 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:02.873462    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j" event={"ID":"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935","Type":"ContainerStarted","Data":"4dcda33c68806c315f30d412c3dcc3e1a0f55bad09d922f2fc1c85fc89356b2c"}
Oct 30 21:46:02.875000 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:02.874974    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" event={"ID":"99b01064-b927-4e4b-9cc9-c5d7dd1d6738","Type":"ContainerStarted","Data":"64cee0cd8d3cabe36bab20145819958fb3978a8f7b8cd7be9a80d6c3ee7ef291"}
Oct 30 21:46:02.875122 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:02.875082    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:46:02.992276 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:02.992239    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" podStartSLOduration=2.146070561 podStartE2EDuration="4.992226951s" totalImagesPullingTime="2.84615639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:02.990547562 +0000 UTC m=+711.268241034" watchObservedRunningTime="2026-10-30 21:46:02.992226951 +0000 UTC m=+711.269920432"
Oct 30 21:46:03.881758 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:03.881716    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j" event={"ID":"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935","Type":"ContainerStarted","Data":"2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56"}
Oct 30 21:46:03.882150 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:03.882045    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:03.998791 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:03.998741    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j" podStartSLOduration=2.209058001 podStartE2EDuration="2.998728657s" totalImagesPullingTime="789.670656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:03.997528564 +0000 UTC m=+712.275222035" watchObservedRunningTime="2026-10-30 21:46:03.998728657 +0000 UTC m=+712.276422127"
Oct 30 21:46:04.862602 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:04.862572    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 21:46:07.785217 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:07.785184    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:46:09.898374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:09.897816    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 21:46:11.747376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.743847    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"]
Oct 30 21:46:11.769373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.761609    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.806843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.806812    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"]
Oct 30 21:46:11.831931 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.831904    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.832080 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.831950    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-metrics-certs\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.832080 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.831971    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6gc5\" (UniqueName: \"kubernetes.io/projected/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-kube-api-access-x6gc5\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.832202 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.832125    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.868373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.864406    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-bc009f-serving-cert\""
Oct 30 21:46:11.933065 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.933032    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6gc5\" (UniqueName: \"kubernetes.io/projected/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-kube-api-access-x6gc5\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.933201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.933126    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.933264 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.933248    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.933313 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:11.933259    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-serving-cert: secret "maas-api-e2e-worker-w3-bc009f-serving-cert" not found
Oct 30 21:46:11.933367 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.933310    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-metrics-certs\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.933367 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:11.933322    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls podName:4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:12.43330685 +0000 UTC m=+720.711000303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls") pod "maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c" (UID: "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2") : secret "maas-api-e2e-worker-w3-bc009f-serving-cert" not found
Oct 30 21:46:11.934064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.934040    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.935717 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.935698    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-metrics-certs\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:11.999885 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:11.999821    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6gc5\" (UniqueName: \"kubernetes.io/projected/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-kube-api-access-x6gc5\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:12.438789 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:12.438757    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:12.441436 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:12.441411    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls\") pod \"maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:12.699994 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:12.699910    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:13.117139 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:13.117107    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"]
Oct 30 21:46:13.118944 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:46:13.118916    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cad39ee_9e75_4c0a_90bf_2f9e4c1fe3e2.slice/crio-7e9b0eec2348d1eea93ce20e381a20ee57daa0a678b5bf9333ee878a636ae286 WatchSource:0}: Error finding container 7e9b0eec2348d1eea93ce20e381a20ee57daa0a678b5bf9333ee878a636ae286: Status 404 returned error can't find the container with id 7e9b0eec2348d1eea93ce20e381a20ee57daa0a678b5bf9333ee878a636ae286
Oct 30 21:46:13.903604 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:13.903575    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 21:46:13.934854 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:13.934825    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c" event={"ID":"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2","Type":"ContainerStarted","Data":"7e9b0eec2348d1eea93ce20e381a20ee57daa0a678b5bf9333ee878a636ae286"}
Oct 30 21:46:14.948398 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:14.943120    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c" event={"ID":"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2","Type":"ContainerStarted","Data":"8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45"}
Oct 30 21:46:14.948398 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:14.943199    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:15.061655 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:15.061604    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c" podStartSLOduration=3.13962546 podStartE2EDuration="4.061589509s" totalImagesPullingTime="921.964049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:15.060822705 +0000 UTC m=+723.338516177" watchObservedRunningTime="2026-10-30 21:46:15.061589509 +0000 UTC m=+723.339282980"
Oct 30 21:46:17.956223 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.956188    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:46:17.960880 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.960855    5561 generic.go:356] "Generic (PLEG): container finished" podID="6ad9b344-5973-422c-ae1b-43df366ee690" containerID="df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5" exitCode=0
Oct 30 21:46:17.961017 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.960926    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" event={"ID":"6ad9b344-5973-422c-ae1b-43df366ee690","Type":"ContainerDied","Data":"df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5"}
Oct 30 21:46:17.961275 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.961258    5561 scope.go:122] "RemoveContainer" containerID="df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5"
Oct 30 21:46:17.962533 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.962494    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:17.962642 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.962577    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:17.962642 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.962622    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:17.962948 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.962927    5561 generic.go:356] "Generic (PLEG): container finished" podID="498354f6-78ca-4228-b51b-f42ad382b63b" containerID="010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d" exitCode=0
Oct 30 21:46:17.963028 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.962990    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" event={"ID":"498354f6-78ca-4228-b51b-f42ad382b63b","Type":"ContainerDied","Data":"010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d"}
Oct 30 21:46:17.963336 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.963316    5561 scope.go:122] "RemoveContainer" containerID="010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d"
Oct 30 21:46:17.964587 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.964559    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:17.964680 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.964642    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:17.964727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:17.964710    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:18.995956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:18.995915    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" event={"ID":"6ad9b344-5973-422c-ae1b-43df366ee690","Type":"ContainerStarted","Data":"fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8"}
Oct 30 21:46:19.000414 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:19.000388    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" event={"ID":"498354f6-78ca-4228-b51b-f42ad382b63b","Type":"ContainerStarted","Data":"fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959"}
Oct 30 21:46:19.157734 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:19.157700    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:46:19.158089 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:19.158004    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:46:19.158235 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:19.158216    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:46:19.472375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:19.471778    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:46:19.472640 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:19.472421    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:46:19.472640 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:19.472479    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:46:20.164375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:20.162650    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:46:20.164375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:20.162705    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:46:20.473430 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:20.469419    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:46:20.473430 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:20.469476    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:46:20.976715 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:20.976683    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 21:46:21.158376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.155701    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:46:21.158376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.155765    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:46:21.469376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.465704    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:46:21.469376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.465762    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:46:21.812636 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.812555    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"]
Oct 30 21:46:21.819065 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.819016    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:21.888370 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.887482    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-c3e2d8-serving-cert\""
Oct 30 21:46:21.888370 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.888143    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"]
Oct 30 21:46:21.927591 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.927562    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:21.927742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.927603    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ccmk\" (UniqueName: \"kubernetes.io/projected/2a77c010-6fce-43c0-a897-77f029793494-kube-api-access-9ccmk\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:21.927742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.927626    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a77c010-6fce-43c0-a897-77f029793494-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:21.927822 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:21.927746    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-metrics-certs\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.029230 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.029200    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.029400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.029234    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ccmk\" (UniqueName: \"kubernetes.io/projected/2a77c010-6fce-43c0-a897-77f029793494-kube-api-access-9ccmk\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.029400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.029256    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a77c010-6fce-43c0-a897-77f029793494-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.029587 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:22.029343    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-serving-cert: secret "maas-api-e2e-worker-w4-c3e2d8-serving-cert" not found
Oct 30 21:46:22.029634 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:22.029625    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls podName:2a77c010-6fce-43c0-a897-77f029793494 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:22.529609786 +0000 UTC m=+730.807303234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls") pod "maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd" (UID: "2a77c010-6fce-43c0-a897-77f029793494") : secret "maas-api-e2e-worker-w4-c3e2d8-serving-cert" not found
Oct 30 21:46:22.029795 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.029781    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-metrics-certs\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.029833 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.029817    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a77c010-6fce-43c0-a897-77f029793494-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.031834 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.031816    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-metrics-certs\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.121032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.120999    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ccmk\" (UniqueName: \"kubernetes.io/projected/2a77c010-6fce-43c0-a897-77f029793494-kube-api-access-9ccmk\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.196437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.196402    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:46:22.196655 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.196460    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:46:22.468832 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.468740    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:46:22.469601 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.469570    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:46:22.533521 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.533482    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.535840 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.535821    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls\") pod \"maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:22.732192 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:22.732112    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:23.024371 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:23.024270    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:46:23.159825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:23.159733    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"]
Oct 30 21:46:23.164884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:23.164457    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:46:23.164884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:23.164507    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:46:23.181742 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:46:23.181688    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a77c010_6fce_43c0_a897_77f029793494.slice/crio-1bbcc9f8b40a0e24a6567102ede34a097aaa104302f4d3f6261a0555acaf3c6d WatchSource:0}: Error finding container 1bbcc9f8b40a0e24a6567102ede34a097aaa104302f4d3f6261a0555acaf3c6d: Status 404 returned error can't find the container with id 1bbcc9f8b40a0e24a6567102ede34a097aaa104302f4d3f6261a0555acaf3c6d
Oct 30 21:46:23.466108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:23.465656    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:46:23.466108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:23.465714    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:46:24.026687 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:24.026648    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd" event={"ID":"2a77c010-6fce-43c0-a897-77f029793494","Type":"ContainerStarted","Data":"1bbcc9f8b40a0e24a6567102ede34a097aaa104302f4d3f6261a0555acaf3c6d"}
Oct 30 21:46:24.223730 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:24.223691    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:46:24.224430 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:24.224386    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:46:24.227014 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:24.226985    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 21:46:24.469114 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:24.469089    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:46:24.469794 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:24.469771    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:46:24.470427 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:24.470406    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 21:46:25.032967 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:25.032923    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd" event={"ID":"2a77c010-6fce-43c0-a897-77f029793494","Type":"ContainerStarted","Data":"1a2213f9abe65552034c1cd7702d4ecfc56df2fbcb9f53cd29bac762f1b9a3ae"}
Oct 30 21:46:25.033394 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:25.033144    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:25.157879 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:25.157830    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd" podStartSLOduration=3.301374783 podStartE2EDuration="4.157815022s" totalImagesPullingTime="856.440239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:25.157275192 +0000 UTC m=+733.434968674" watchObservedRunningTime="2026-10-30 21:46:25.157815022 +0000 UTC m=+733.435508492"
Oct 30 21:46:27.018902 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.018515    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"]
Oct 30 21:46:27.022653 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.022630    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.100390 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.100340    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.100528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.100463    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88b0307-6421-42cb-9bf4-8cef89e95af7-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.100528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.100520    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxwvz\" (UniqueName: \"kubernetes.io/projected/d88b0307-6421-42cb-9bf4-8cef89e95af7-kube-api-access-xxwvz\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.100623 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.100602    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-metrics-certs\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.120599 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.120572    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"]
Oct 30 21:46:27.129757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.129734    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-9b51e1-serving-cert\""
Oct 30 21:46:27.201218 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.201193    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxwvz\" (UniqueName: \"kubernetes.io/projected/d88b0307-6421-42cb-9bf4-8cef89e95af7-kube-api-access-xxwvz\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.201389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.201345    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-metrics-certs\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.201509 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.201443    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.201563 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.201527    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88b0307-6421-42cb-9bf4-8cef89e95af7-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.201607 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:27.201571    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-serving-cert: secret "maas-api-e2e-worker-w5-9b51e1-serving-cert" not found
Oct 30 21:46:27.201653 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:27.201622    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls podName:d88b0307-6421-42cb-9bf4-8cef89e95af7 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:27.701608922 +0000 UTC m=+735.979302371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls") pod "maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck" (UID: "d88b0307-6421-42cb-9bf4-8cef89e95af7") : secret "maas-api-e2e-worker-w5-9b51e1-serving-cert" not found
Oct 30 21:46:27.202141 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.202118    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88b0307-6421-42cb-9bf4-8cef89e95af7-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.203718 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.203696    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-metrics-certs\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.270765 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.270691    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxwvz\" (UniqueName: \"kubernetes.io/projected/d88b0307-6421-42cb-9bf4-8cef89e95af7-kube-api-access-xxwvz\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.708669 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.708634    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.712618 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.712582    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls\") pod \"maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:27.856448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.856418    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:46:27.936280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:27.936224    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:28.320697 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:28.320670    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"]
Oct 30 21:46:28.322339 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:46:28.322313    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88b0307_6421_42cb_9bf4_8cef89e95af7.slice/crio-b0733204195d0f0236cfe940617678d9b5e7dec1bf8a321e66020b6ac32a3140 WatchSource:0}: Error finding container b0733204195d0f0236cfe940617678d9b5e7dec1bf8a321e66020b6ac32a3140: Status 404 returned error can't find the container with id b0733204195d0f0236cfe940617678d9b5e7dec1bf8a321e66020b6ac32a3140
Oct 30 21:46:29.105742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:29.105711    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck" event={"ID":"d88b0307-6421-42cb-9bf4-8cef89e95af7","Type":"ContainerStarted","Data":"b0733204195d0f0236cfe940617678d9b5e7dec1bf8a321e66020b6ac32a3140"}
Oct 30 21:46:30.116716 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:30.116672    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck" event={"ID":"d88b0307-6421-42cb-9bf4-8cef89e95af7","Type":"ContainerStarted","Data":"233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae"}
Oct 30 21:46:30.117598 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:30.117570    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:30.244621 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:30.241454    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck" podStartSLOduration=3.4486487710000002 podStartE2EDuration="4.241435811s" totalImagesPullingTime="792.78704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:30.238493393 +0000 UTC m=+738.516186864" watchObservedRunningTime="2026-10-30 21:46:30.241435811 +0000 UTC m=+738.519129283"
Oct 30 21:46:31.043840 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:31.043812    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 21:46:37.131971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:37.131940    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 21:46:49.221959 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:49.221922    5561 generic.go:356] "Generic (PLEG): container finished" podID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerID="b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4" exitCode=0
Oct 30 21:46:49.222472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:49.222000    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" event={"ID":"e1c0dffe-c267-4356-aa24-03b78c24dd49","Type":"ContainerDied","Data":"b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4"}
Oct 30 21:46:49.222472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:49.222371    5561 scope.go:122] "RemoveContainer" containerID="b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4"
Oct 30 21:46:49.223668 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:49.223641    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:49.223733 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:49.223716    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:49.223769 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:49.223752    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:49.779374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:49.778859    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:46:50.228839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:50.228807    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" event={"ID":"e1c0dffe-c267-4356-aa24-03b78c24dd49","Type":"ContainerStarted","Data":"6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b"}
Oct 30 21:46:50.414917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:50.414885    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:46:50.415086 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:50.415057    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:46:50.415136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:50.415102    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:46:51.415346 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:51.415305    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:46:51.415743 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:51.415386    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:46:52.415818 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:52.415781    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:46:52.416281 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:52.415838    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:46:52.994953 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:46:52.994909    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:46:53.414980 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:53.414946    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:46:53.415156 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:53.415000    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:46:54.414916 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:54.414881    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:46:54.415284 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:54.414930    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:46:55.446040 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:55.446005    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:46:55.447189 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:55.447159    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:46:55.448137 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:46:55.448116    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 21:47:03.572557 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:03.572522    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:47:18.188801 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:18.188770    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:47:22.980318 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:47:22.980283    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:47:42.845760 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:42.845722    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:47:52.981227 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:47:52.981149    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:47:55.400989 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.400957    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"]
Oct 30 21:47:55.404709 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.404693    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489100 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489070    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489220 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489109    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489220 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489141    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de424a99-eb6c-40c9-95d3-d048c456cb53-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489298 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489240    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489298 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489273    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489397 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489370    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489450 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489438    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489466    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.489520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.489490    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-744sg\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-kube-api-access-744sg\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.508099 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.508061    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-557708-openshift-default-dockercfg-87jdg\""
Oct 30 21:47:55.510138 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.510113    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"]
Oct 30 21:47:55.590261 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590236    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590411 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590275    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590420    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de424a99-eb6c-40c9-95d3-d048c456cb53-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590472    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590539    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590588    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590706 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590636    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590706 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590656    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590706 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590687    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590706 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590700    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.590944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590918    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-744sg\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-kube-api-access-744sg\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.591028 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.590926    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.591119 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.591098    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de424a99-eb6c-40c9-95d3-d048c456cb53-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.591173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.591123    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.592884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.592864    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.593100 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.593083    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.649074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.649046    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.649262 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.649244    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-744sg\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-kube-api-access-744sg\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:55.717511 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:55.717457    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:56.146774 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.146745    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"]
Oct 30 21:47:56.147896 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:47:56.147868    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde424a99_eb6c_40c9_95d3_d048c456cb53.slice/crio-c6768433a09f6472bc22596b49c27112f4d1a60af21fbde252bb872cd321821f WatchSource:0}: Error finding container c6768433a09f6472bc22596b49c27112f4d1a60af21fbde252bb872cd321821f: Status 404 returned error can't find the container with id c6768433a09f6472bc22596b49c27112f4d1a60af21fbde252bb872cd321821f
Oct 30 21:47:56.150343 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.150309    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:56.150445 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.150411    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:56.150485 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.150451    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:56.582062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.581980    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" event={"ID":"de424a99-eb6c-40c9-95d3-d048c456cb53","Type":"ContainerStarted","Data":"30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267"}
Oct 30 21:47:56.582062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.582033    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" event={"ID":"de424a99-eb6c-40c9-95d3-d048c456cb53","Type":"ContainerStarted","Data":"c6768433a09f6472bc22596b49c27112f4d1a60af21fbde252bb872cd321821f"}
Oct 30 21:47:56.714184 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.714130    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podStartSLOduration=1.7141158939999999 podStartE2EDuration="1.714115894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:56.712335231 +0000 UTC m=+824.990028703" watchObservedRunningTime="2026-10-30 21:47:56.714115894 +0000 UTC m=+824.991809364"
Oct 30 21:47:56.718088 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:56.718062    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:47:57.719584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:57.719548    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:47:57.719980 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:57.719598    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 30 21:47:58.493025 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.492994    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"]
Oct 30 21:47:58.496898 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.496875    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.521731 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.521702    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8317276a-8423-494a-8fec-49eb1d43ac3c-tls-certs\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.521835 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.521759    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-model-cache\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.521881 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.521838    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xdcp\" (UniqueName: \"kubernetes.io/projected/8317276a-8423-494a-8fec-49eb1d43ac3c-kube-api-access-9xdcp\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.521924 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.521900    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-dshm\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.521965 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.521920    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-tmp-dir\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.521965 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.521957    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-home\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.583072 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.583029    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"]
Oct 30 21:47:58.604181 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.604153    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"openshift-service-ca.crt\""
Oct 30 21:47:58.604319 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.604209    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"kube-root-ca.crt\""
Oct 30 21:47:58.604380 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.604309    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"simulator-w0-dda9f5-kserve-self-signed-certs\""
Oct 30 21:47:58.604439 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.604414    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"default-dockercfg-47mqx\""
Oct 30 21:47:58.622637 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.622612    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-dshm\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.622637 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.622640    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-tmp-dir\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.622825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.622664    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-home\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.622825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.622768    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8317276a-8423-494a-8fec-49eb1d43ac3c-tls-certs\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.622952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.622850    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-model-cache\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.622952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.622890    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xdcp\" (UniqueName: \"kubernetes.io/projected/8317276a-8423-494a-8fec-49eb1d43ac3c-kube-api-access-9xdcp\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.623120 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.623097    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-tmp-dir\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.623120 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.623113    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-home\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.623394 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.623340    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-model-cache\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.625015 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.624998    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-dshm\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.625241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.625225    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8317276a-8423-494a-8fec-49eb1d43ac3c-tls-certs\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.698039 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.697914    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xdcp\" (UniqueName: \"kubernetes.io/projected/8317276a-8423-494a-8fec-49eb1d43ac3c-kube-api-access-9xdcp\") pod \"simulator-w0-dda9f5-kserve-77f5db79db-jzqpq\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") " pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:58.717935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.717906    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:47:58.718074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.717953    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 30 21:47:58.809451 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:58.809379    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:47:59.202580 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:59.202555    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"]
Oct 30 21:47:59.204695 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:47:59.204665    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8317276a_8423_494a_8fec_49eb1d43ac3c.slice/crio-327938a30d704f5e6add1fb579f198fcf36baf839d020653230bb2ad5365222f WatchSource:0}: Error finding container 327938a30d704f5e6add1fb579f198fcf36baf839d020653230bb2ad5365222f: Status 404 returned error can't find the container with id 327938a30d704f5e6add1fb579f198fcf36baf839d020653230bb2ad5365222f
Oct 30 21:47:59.599910 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:59.599826    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" event={"ID":"8317276a-8423-494a-8fec-49eb1d43ac3c","Type":"ContainerStarted","Data":"3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a"}
Oct 30 21:47:59.599910 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:59.599878    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" event={"ID":"8317276a-8423-494a-8fec-49eb1d43ac3c","Type":"ContainerStarted","Data":"327938a30d704f5e6add1fb579f198fcf36baf839d020653230bb2ad5365222f"}
Oct 30 21:47:59.718664 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:59.718633    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:47:59.718818 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:59.718681    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 30 21:47:59.724997 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:59.724954    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podStartSLOduration=1.724940226 podStartE2EDuration="1.724940226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:59.723699277 +0000 UTC m=+828.001392740" watchObservedRunningTime="2026-10-30 21:47:59.724940226 +0000 UTC m=+828.002633696"
Oct 30 21:47:59.809771 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:47:59.809740    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:48:00.718205 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:00.718178    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:48:00.718371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:00.718222    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 30 21:48:01.717899 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:01.717867    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:48:01.718277 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:01.717913    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 30 21:48:02.807808 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:02.807774    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:48:02.808294 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:02.808152    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:48:02.809852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:02.809825    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 21:48:07.704419 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:07.704385    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:48:08.809740 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:08.809709    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:48:08.822195 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:08.822173    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:48:09.669547 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:09.669517    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:48:22.979191 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:48:22.979144    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:48:29.559899 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.559864    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"]
Oct 30 21:48:29.564467 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.564442    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.622181 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.622151    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c79cb5\"/\"kube-root-ca.crt\""
Oct 30 21:48:29.623558 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.623532    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"]
Oct 30 21:48:29.649421 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.649397    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-home\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.649552 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.649531    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daeb4d95-948f-48a5-9c07-38495186017a-tls-certs\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.649606 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.649560    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxcwc\" (UniqueName: \"kubernetes.io/projected/daeb4d95-948f-48a5-9c07-38495186017a-kube-api-access-dxcwc\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.649665 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.649611    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-model-cache\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.649737 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.649721    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-dshm\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.649775 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.649747    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-tmp-dir\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.671614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.671591    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c79cb5\"/\"openshift-service-ca.crt\""
Oct 30 21:48:29.672761 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.672747    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c79cb5\"/\"default-dockercfg-sbmpt\""
Oct 30 21:48:29.672761 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.672756    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c79cb5\"/\"premium-w2-c79cb5-kserve-self-signed-certs\""
Oct 30 21:48:29.751057 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751021    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daeb4d95-948f-48a5-9c07-38495186017a-tls-certs\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751057 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751055    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxcwc\" (UniqueName: \"kubernetes.io/projected/daeb4d95-948f-48a5-9c07-38495186017a-kube-api-access-dxcwc\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751268 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751096    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-model-cache\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751268 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751255    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-dshm\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751358 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751290    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-tmp-dir\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751404 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751341    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-home\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751579 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751563    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-model-cache\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751657 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751641    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-tmp-dir\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.751796 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.751766    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-home\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.753481 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.753454    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-dshm\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.753668 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.753654    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daeb4d95-948f-48a5-9c07-38495186017a-tls-certs\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.810877 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.810813    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxcwc\" (UniqueName: \"kubernetes.io/projected/daeb4d95-948f-48a5-9c07-38495186017a-kube-api-access-dxcwc\") pod \"premium-w2-c79cb5-kserve-6bd68df849-t5qd9\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") " pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:29.876268 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:29.876248    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:30.282521 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:30.282500    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"]
Oct 30 21:48:30.283856 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:48:30.283829    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaeb4d95_948f_48a5_9c07_38495186017a.slice/crio-b05078ec8c5db8a2360b50f11ad711c12653cc2e78848dbe58e14a3b600a9fdd WatchSource:0}: Error finding container b05078ec8c5db8a2360b50f11ad711c12653cc2e78848dbe58e14a3b600a9fdd: Status 404 returned error can't find the container with id b05078ec8c5db8a2360b50f11ad711c12653cc2e78848dbe58e14a3b600a9fdd
Oct 30 21:48:30.771828 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:30.771791    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" event={"ID":"daeb4d95-948f-48a5-9c07-38495186017a","Type":"ContainerStarted","Data":"cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564"}
Oct 30 21:48:30.772204 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:30.771837    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" event={"ID":"daeb4d95-948f-48a5-9c07-38495186017a","Type":"ContainerStarted","Data":"b05078ec8c5db8a2360b50f11ad711c12653cc2e78848dbe58e14a3b600a9fdd"}
Oct 30 21:48:30.877384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:30.877342    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:30.889494 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:30.889451    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podStartSLOduration=1.889439206 podStartE2EDuration="1.889439206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:30.889383222 +0000 UTC m=+859.167076693" watchObservedRunningTime="2026-10-30 21:48:30.889439206 +0000 UTC m=+859.167132675"
Oct 30 21:48:37.060869 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:37.060838    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:48:39.876955 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:39.876923    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:39.889141 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:39.889119    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:40.832882 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:40.832855    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:48:52.981200 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:48:52.981167    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:48:55.190189 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:48:55.190151    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" podUID="9a9ea5e58d330910c40e8b399ca003fd" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 21:49:07.266271 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:07.266241    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:49:11.940870 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:11.940843    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:49:11.948296 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:11.948269    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:49:22.980571 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:49:22.980486    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:49:28.699153 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:28.699110    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"]
Oct 30 21:49:28.699649 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:28.699367    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy" containerID="cri-o://93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc" gracePeriod=30
Oct 30 21:49:33.639673 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:33.639640    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:49:33.953545 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:33.953521    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:49:34.040542 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040506    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-socket\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.040754 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040608    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-data\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.040754 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040678    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-podinfo\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.040754 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040724    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-envoy\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.040939 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040760    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-certs\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.040939 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040782    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-credential-socket\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.040939 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040861    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-kube-api-access-9z7sw\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-kube-api-access-9z7sw\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.041096 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040935    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-token\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-token\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.041096 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.040967    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istiod-ca-cert\") pod \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\" (UID: \"8ef00f19-fa1a-4a53-af23-534b63ed24a2\") "
Oct 30 21:49:34.042668 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.041754    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istiod-ca-cert" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:34.042668 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.042096    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-socket" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:34.046513 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.043656    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-credential-socket" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:34.046513 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.045834    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-data" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:34.047299 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.047261    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-certs" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:34.059368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.057115    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-envoy" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:34.066971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.062712    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-token" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:34.066971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.063051    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-podinfo" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:34.069237 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.069213    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-kube-api-access-9z7sw" pod "8ef00f19-fa1a-4a53-af23-534b63ed24a2" (UID: "8ef00f19-fa1a-4a53-af23-534b63ed24a2"). InnerVolumeSpecName "kube-api-access-9z7sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:34.100038 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.100004    5561 generic.go:356] "Generic (PLEG): container finished" podID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerID="93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc" exitCode=0
Oct 30 21:49:34.100201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.100059    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"
Oct 30 21:49:34.100201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.100090    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" event={"ID":"8ef00f19-fa1a-4a53-af23-534b63ed24a2","Type":"ContainerDied","Data":"93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc"}
Oct 30 21:49:34.100201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.100158    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq" event={"ID":"8ef00f19-fa1a-4a53-af23-534b63ed24a2","Type":"ContainerDied","Data":"cc6e16e4abbd438c831643b1d52a407e6883b73d56e22b3e1ba41b46e13b1a19"}
Oct 30 21:49:34.100201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.100182    5561 scope.go:122] "RemoveContainer" containerID="93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc"
Oct 30 21:49:34.112768 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.112745    5561 scope.go:122] "RemoveContainer" containerID="93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc"
Oct 30 21:49:34.113200 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:49:34.113128    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc\": container with ID starting with 93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc not found: ID does not exist" containerID="93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc"
Oct 30 21:49:34.113200 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.113173    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc"} err="failed to get container status \"93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc\": rpc error: code = NotFound desc = could not find container \"93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc\": container with ID starting with 93ba75a8c63a58dcd0c474752779ac169388922c4d7f396f9b15fbc6e8dd6cfc not found: ID does not exist"
Oct 30 21:49:34.142851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142822    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142858    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142874    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142888    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142902    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142917    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142930    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142944    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ef00f19-fa1a-4a53-af23-534b63ed24a2-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.142971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.142958    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z7sw\" (UniqueName: \"kubernetes.io/projected/8ef00f19-fa1a-4a53-af23-534b63ed24a2-kube-api-access-9z7sw\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.238993 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.238930    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"]
Oct 30 21:49:34.245408 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:34.245383    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq"]
Oct 30 21:49:35.845343 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:35.845312    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" path="/var/lib/kubelet/pods/8ef00f19-fa1a-4a53-af23-534b63ed24a2/volumes"
Oct 30 21:49:40.737961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.737924    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"]
Oct 30 21:49:40.738660 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.738639    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy"
Oct 30 21:49:40.738726 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.738664    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy"
Oct 30 21:49:40.738806 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.738794    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ef00f19-fa1a-4a53-af23-534b63ed24a2" containerName="istio-proxy"
Oct 30 21:49:40.741946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.741930    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.809921 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.809884    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pzg5\" (UniqueName: \"kubernetes.io/projected/f6bed758-9227-46c9-ba76-48b1ca055dae-kube-api-access-2pzg5\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.810095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.810023    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-tmp-dir\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.810095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.810055    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-home\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.810095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.810078    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-model-cache\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.810282 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.810193    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-dshm\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.810282 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.810241    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bed758-9227-46c9-ba76-48b1ca055dae-tls-certs\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.845704 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.845676    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"kube-root-ca.crt\""
Oct 30 21:49:40.845884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.845860    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"openshift-service-ca.crt\""
Oct 30 21:49:40.846005 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.845973    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"default-dockercfg-kt4rt\""
Oct 30 21:49:40.847861 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.847840    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"]
Oct 30 21:49:40.849023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.849008    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"premium-w5-9b51e1-kserve-self-signed-certs\""
Oct 30 21:49:40.911886 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.911851    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-dshm\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912065 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.911935    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bed758-9227-46c9-ba76-48b1ca055dae-tls-certs\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912065 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.911999    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pzg5\" (UniqueName: \"kubernetes.io/projected/f6bed758-9227-46c9-ba76-48b1ca055dae-kube-api-access-2pzg5\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912227 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.912205    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-tmp-dir\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912295 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.912242    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-home\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912295 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.912279    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-model-cache\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912617 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.912590    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-tmp-dir\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912712 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.912599    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-model-cache\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.912750 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.912721    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-home\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.914294 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.914269    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-dshm\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.914594 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.914578    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bed758-9227-46c9-ba76-48b1ca055dae-tls-certs\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:40.969983 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:40.969959    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pzg5\" (UniqueName: \"kubernetes.io/projected/f6bed758-9227-46c9-ba76-48b1ca055dae-kube-api-access-2pzg5\") pod \"premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") " pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:41.053652 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:41.053583    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:41.481683 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:41.478000    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"]
Oct 30 21:49:41.485259 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:49:41.485211    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6bed758_9227_46c9_ba76_48b1ca055dae.slice/crio-faff16ff4293c4d4b5557fa01ff43a335f6cac65183a0ec926655d00256ed813 WatchSource:0}: Error finding container faff16ff4293c4d4b5557fa01ff43a335f6cac65183a0ec926655d00256ed813: Status 404 returned error can't find the container with id faff16ff4293c4d4b5557fa01ff43a335f6cac65183a0ec926655d00256ed813
Oct 30 21:49:42.144953 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:42.144909    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" event={"ID":"f6bed758-9227-46c9-ba76-48b1ca055dae","Type":"ContainerStarted","Data":"40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c"}
Oct 30 21:49:42.145389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:42.144966    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" event={"ID":"f6bed758-9227-46c9-ba76-48b1ca055dae","Type":"ContainerStarted","Data":"faff16ff4293c4d4b5557fa01ff43a335f6cac65183a0ec926655d00256ed813"}
Oct 30 21:49:42.262675 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:42.262606    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podStartSLOduration=2.262588288 podStartE2EDuration="2.262588288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:42.260457466 +0000 UTC m=+930.538150939" watchObservedRunningTime="2026-10-30 21:49:42.262588288 +0000 UTC m=+930.540281760"
Oct 30 21:49:43.053861 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:43.053832    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:51.053741 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:51.053705    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:51.068463 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:51.068441    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:51.200757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:49:51.200728    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:49:52.981393 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:49:52.980756    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:50:07.378640 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:07.378602    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:50:22.988864 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:50:22.988817    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:50:52.980885 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:50:52.980802    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:50:54.422040 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.422006    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"]
Oct 30 21:50:54.425711 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.425696    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.477008 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.476974    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-home\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.477159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.477022    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-model-cache\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.477256 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.477229    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9307b981-13a4-4e6a-836d-e9d0911502ba-tls-certs\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.477322 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.477287    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-tmp-dir\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.477429 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.477407    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc6zc\" (UniqueName: \"kubernetes.io/projected/9307b981-13a4-4e6a-836d-e9d0911502ba-kube-api-access-wc6zc\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.477492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.477464    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-dshm\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.485285 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.485259    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"]
Oct 30 21:50:54.532536 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.532514    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-bc009f\"/\"openshift-service-ca.crt\""
Oct 30 21:50:54.532667 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.532621    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-bc009f\"/\"kube-root-ca.crt\""
Oct 30 21:50:54.533771 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.533752    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bc009f\"/\"default-dockercfg-hz59j\""
Oct 30 21:50:54.533857 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.533778    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bc009f\"/\"distinct-w3-bc009f-kserve-self-signed-certs\""
Oct 30 21:50:54.578855 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.578829    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9307b981-13a4-4e6a-836d-e9d0911502ba-tls-certs\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.578946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.578872    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-tmp-dir\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.578946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.578913    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc6zc\" (UniqueName: \"kubernetes.io/projected/9307b981-13a4-4e6a-836d-e9d0911502ba-kube-api-access-wc6zc\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.579048 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.578967    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-dshm\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.579048 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.579025    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-home\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.579048 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.579044    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-model-cache\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.579260 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.579241    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-tmp-dir\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.579422 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.579404    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-model-cache\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.579488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.579423    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-home\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.581147 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.581128    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-dshm\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.581435 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.581419    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9307b981-13a4-4e6a-836d-e9d0911502ba-tls-certs\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.640392 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.640331    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc6zc\" (UniqueName: \"kubernetes.io/projected/9307b981-13a4-4e6a-836d-e9d0911502ba-kube-api-access-wc6zc\") pod \"distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:54.736724 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:54.736645    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:55.113769 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:55.113745    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"]
Oct 30 21:50:55.115626 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:50:55.115593    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9307b981_13a4_4e6a_836d_e9d0911502ba.slice/crio-26650fc7ddac0760ff69f189c4654601bee21c2080c57f71d0eda3eb4377bf15 WatchSource:0}: Error finding container 26650fc7ddac0760ff69f189c4654601bee21c2080c57f71d0eda3eb4377bf15: Status 404 returned error can't find the container with id 26650fc7ddac0760ff69f189c4654601bee21c2080c57f71d0eda3eb4377bf15
Oct 30 21:50:55.559081 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:55.559046    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" event={"ID":"9307b981-13a4-4e6a-836d-e9d0911502ba","Type":"ContainerStarted","Data":"f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d"}
Oct 30 21:50:55.559081 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:55.559094    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" event={"ID":"9307b981-13a4-4e6a-836d-e9d0911502ba","Type":"ContainerStarted","Data":"26650fc7ddac0760ff69f189c4654601bee21c2080c57f71d0eda3eb4377bf15"}
Oct 30 21:50:55.677669 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:55.677606    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podStartSLOduration=1.677592883 podStartE2EDuration="1.677592883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:55.675939797 +0000 UTC m=+1003.953633271" watchObservedRunningTime="2026-10-30 21:50:55.677592883 +0000 UTC m=+1003.955286407"
Oct 30 21:50:55.737220 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:55.737192    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:50:56.823019 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.822986    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"]
Oct 30 21:50:56.826741 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.826723    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.904873 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.904840    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905007 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.904885    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905007 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.904952    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fsvc\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-kube-api-access-8fsvc\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.905026    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.905064    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a75d0b-977e-4970-81b9-b6665bbeb6db-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905235 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.905101    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905235 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.905140    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905312 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.905239    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.905312 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.905273    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:56.932225 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:56.932198    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"]
Oct 30 21:50:57.006718 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.006692    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.006718 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.006725    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.006916 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.006750    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.006916 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.006769    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fsvc\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-kube-api-access-8fsvc\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.006916 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.006810    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007080 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.006984    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a75d0b-977e-4970-81b9-b6665bbeb6db-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007080 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007050    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007094    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007159    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007174    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007345 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007231    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007474 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007452    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007568 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007548    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.007709 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.007690    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a75d0b-977e-4970-81b9-b6665bbeb6db-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.009285 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.009265    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.009373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.009285    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.063922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.063899    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.064047 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.064029    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fsvc\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-kube-api-access-8fsvc\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.140256 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.140238    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:57.569137 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.569036    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"]
Oct 30 21:50:57.571660 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:50:57.571630    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a75d0b_977e_4970_81b9_b6665bbeb6db.slice/crio-71fc0f95f9d43c11abcf1fa585f28b21e6e9b751153131d3358905295a4846d0 WatchSource:0}: Error finding container 71fc0f95f9d43c11abcf1fa585f28b21e6e9b751153131d3358905295a4846d0: Status 404 returned error can't find the container with id 71fc0f95f9d43c11abcf1fa585f28b21e6e9b751153131d3358905295a4846d0
Oct 30 21:50:57.574339 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.574305    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:57.574472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.574389    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:57.574472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.574421    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:57.757925 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:57.757882    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:50:58.067549 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.067503    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"]
Oct 30 21:50:58.072864 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.072833    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.148742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.148706    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"]
Oct 30 21:50:58.182417 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.180437    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-9b51e1-openshift-default-dockercfg-b9z85\""
Oct 30 21:50:58.223778 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.223748    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.223985 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.223970    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s66z\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-kube-api-access-8s66z\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.224079 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.224069    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.224158 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.224141    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec13631d-151c-4fec-b662-34280fc56a06-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.224263 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.224254    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.224381 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.224371    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec13631d-151c-4fec-b662-34280fc56a06-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.224488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.224479    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.224593 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.224584    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.224700 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.224691    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326408 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326311    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326408 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326392    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec13631d-151c-4fec-b662-34280fc56a06-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326631 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326427    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326631 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326495    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326631 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326554    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326631 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326589    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326631 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326613    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s66z\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-kube-api-access-8s66z\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326870 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326665    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326870 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326716    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec13631d-151c-4fec-b662-34280fc56a06-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.326870 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.326810    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.327128 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.327096    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.327190 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.327129    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.327190 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.327156    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.327425 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.327335    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec13631d-151c-4fec-b662-34280fc56a06-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.329166 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.329141    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.329318 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.329295    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec13631d-151c-4fec-b662-34280fc56a06-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.415026 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.414946    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.417117 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.417085    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s66z\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-kube-api-access-8s66z\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:58.579627 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.579527    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" event={"ID":"37a75d0b-977e-4970-81b9-b6665bbeb6db","Type":"ContainerStarted","Data":"c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b"}
Oct 30 21:50:58.579627 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.579593    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" event={"ID":"37a75d0b-977e-4970-81b9-b6665bbeb6db","Type":"ContainerStarted","Data":"71fc0f95f9d43c11abcf1fa585f28b21e6e9b751153131d3358905295a4846d0"}
Oct 30 21:50:58.693880 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:58.693851    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:59.141100 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.141066    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:50:59.142578 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.142544    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:50:59.142725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.142601    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:50:59.207151 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.207055    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podStartSLOduration=3.207038262 podStartE2EDuration="3.207038262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:58.729620362 +0000 UTC m=+1007.007313830" watchObservedRunningTime="2026-10-30 21:50:59.207038262 +0000 UTC m=+1007.484731735"
Oct 30 21:50:59.211583 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.211554    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"]
Oct 30 21:50:59.216961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.216920    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:59.217063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.217005    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:59.217063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.217050    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:59.594923 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.594828    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" event={"ID":"ec13631d-151c-4fec-b662-34280fc56a06","Type":"ContainerStarted","Data":"26238087ddd5c3c3b9749829c09ff4af9dd0dabf05495e40fe1f484b6a6e20f5"}
Oct 30 21:50:59.594923 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.594896    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" event={"ID":"ec13631d-151c-4fec-b662-34280fc56a06","Type":"ContainerStarted","Data":"278e27a7980d199bbdedee2cce46a71977b06e2db0ac4632483fc430408af2cb"}
Oct 30 21:50:59.696368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.694428    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:50:59.730756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:50:59.730561    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podStartSLOduration=1.730543646 podStartE2EDuration="1.730543646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:59.728661848 +0000 UTC m=+1008.006355318" watchObservedRunningTime="2026-10-30 21:50:59.730543646 +0000 UTC m=+1008.008237118"
Oct 30 21:51:00.141085 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:00.141049    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:51:00.141270 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:00.141101    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:51:00.695977 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:00.695930    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:51:00.696153 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:00.695993    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:51:01.141181 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:01.141149    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:51:01.141346 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:01.141199    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:51:01.697378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:01.696298    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:51:01.697378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:01.696386    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:51:02.141363 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:02.141328    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:51:02.141748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:02.141397    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:51:02.695391 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:02.694781    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:51:02.695391 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:02.694844    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:51:03.141191 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:03.141160    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:51:03.141383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:03.141212    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:51:03.694582 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:03.694549    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:51:03.694752 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:03.694600    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:51:04.694673 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:04.694639    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:51:04.695136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:04.694700    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:51:04.737491 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:04.737462    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:51:04.777958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:04.777929    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:51:05.182531 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.182487    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:51:05.182725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.182554    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:51:05.411550 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.411518    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:51:05.411916 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.411887    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:51:05.413418 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.413338    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 21:51:05.651189 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.651160    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:51:05.694809 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.694773    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:51:05.695254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:05.694837    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:51:06.698908 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:06.698873    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:51:06.699336 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:06.699317    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:51:06.699971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:06.699952    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 21:51:22.982164 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:51:22.982113    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:51:29.660339 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:29.660297    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"]
Oct 30 21:51:29.660964 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:29.660517    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy" containerID="cri-o://f22a8b47ca235f2918a4ecdc3db117d2fd7847e1a02f232a6b7f1911e823ee6b" gracePeriod=30
Oct 30 21:51:34.809110 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.809079    5561 generic.go:356] "Generic (PLEG): container finished" podID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerID="f22a8b47ca235f2918a4ecdc3db117d2fd7847e1a02f232a6b7f1911e823ee6b" exitCode=0
Oct 30 21:51:34.809465 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.809125    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" event={"ID":"93de0bd7-f0de-46c2-94cb-d18f6836da70","Type":"ContainerDied","Data":"f22a8b47ca235f2918a4ecdc3db117d2fd7847e1a02f232a6b7f1911e823ee6b"}
Oct 30 21:51:34.917378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.917333    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:51:34.953584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953527    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-data\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.953584 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953571    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-certs\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.953744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953606    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93de0bd7-f0de-46c2-94cb-d18f6836da70-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de0bd7-f0de-46c2-94cb-d18f6836da70-istiod-ca-cert\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.953744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953624    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-podinfo\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.953744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953648    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-socket\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.953744 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953694    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-kube-api-access-dnb7z\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-kube-api-access-dnb7z\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.954067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953752    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-envoy\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.954067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953789    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-data" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:34.954067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953797    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-credential-socket\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.954067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953934    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-certs" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:34.954067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953978    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-token\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-token\") pod \"93de0bd7-f0de-46c2-94cb-d18f6836da70\" (UID: \"93de0bd7-f0de-46c2-94cb-d18f6836da70\") "
Oct 30 21:51:34.954067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.953993    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-socket" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:34.954343 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.954068    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-credential-socket" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:34.954343 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.954141    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93de0bd7-f0de-46c2-94cb-d18f6836da70-istiod-ca-cert" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:34.954573 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.954554    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.954644 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.954577    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.954644 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.954592    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de0bd7-f0de-46c2-94cb-d18f6836da70-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.954644 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.954605    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.954644 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.954620    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.955864 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.955839    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-podinfo" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:34.955864 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.955850    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-kube-api-access-dnb7z" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "kube-api-access-dnb7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:34.956311 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.956280    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-envoy" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:34.956311 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:34.956298    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-token" pod "93de0bd7-f0de-46c2-94cb-d18f6836da70" (UID: "93de0bd7-f0de-46c2-94cb-d18f6836da70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:35.055475 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:35.055447    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:35.055475 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:35.055473    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:35.055622 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:35.055483    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de0bd7-f0de-46c2-94cb-d18f6836da70-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:35.055622 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:35.055493    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnb7z\" (UniqueName: \"kubernetes.io/projected/93de0bd7-f0de-46c2-94cb-d18f6836da70-kube-api-access-dnb7z\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:35.815695 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:35.815666    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"
Oct 30 21:51:35.816079 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:35.815708    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq" event={"ID":"93de0bd7-f0de-46c2-94cb-d18f6836da70","Type":"ContainerDied","Data":"da54fdf1b0e59921904794b31b2a49072329907e12b103ed8a780f8dd7470782"}
Oct 30 21:51:35.816079 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:35.815789    5561 scope.go:122] "RemoveContainer" containerID="f22a8b47ca235f2918a4ecdc3db117d2fd7847e1a02f232a6b7f1911e823ee6b"
Oct 30 21:51:36.033533 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:36.033498    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"]
Oct 30 21:51:36.084451 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:36.084373    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq"]
Oct 30 21:51:37.845443 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:37.845410    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" path="/var/lib/kubelet/pods/93de0bd7-f0de-46c2-94cb-d18f6836da70/volumes"
Oct 30 21:51:38.234735 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:38.234697    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:51:43.472431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.472395    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"]
Oct 30 21:51:43.472901 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.472887    5561 cpu_manager.go:414] "containerMap: removing container" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy"
Oct 30 21:51:43.472943 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.472902    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy"
Oct 30 21:51:43.472997 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.472988    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="93de0bd7-f0de-46c2-94cb-d18f6836da70" containerName="istio-proxy"
Oct 30 21:51:43.477556 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.477538    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.539600 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.539563    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"]
Oct 30 21:51:43.560582 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.560557    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.560697 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.560587    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.560993 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.560917    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.561087 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.561052    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b174d9-930f-4553-b456-cf975d3612f8-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.561142 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.561089    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b174d9-930f-4553-b456-cf975d3612f8-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.561226 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.561208    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.561286 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.561269    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.561389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.561368    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hpbg\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-kube-api-access-5hpbg\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.561435 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.561423    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662402 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662337    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hpbg\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-kube-api-access-5hpbg\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662563 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662494    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662563 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662532    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662563 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662558    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662729 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662680    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662783 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662755    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b174d9-930f-4553-b456-cf975d3612f8-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662836 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662789    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b174d9-930f-4553-b456-cf975d3612f8-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662891 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662858    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662918    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.662944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662927    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.663055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.662968    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.663055 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.663021    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.663297 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.663272    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.663634 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.663613    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b174d9-930f-4553-b456-cf975d3612f8-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.665166 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.665143    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.665337 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.665316    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b174d9-930f-4553-b456-cf975d3612f8-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.717708 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.717682    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"]
Oct 30 21:51:43.722725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.722661    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.722725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.722709    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hpbg\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-kube-api-access-5hpbg\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.723316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.723299    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.775907 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.775884    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-bc009f-openshift-default-dockercfg-4jwnk\""
Oct 30 21:51:43.783773 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.783747    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"]
Oct 30 21:51:43.790026 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.789999    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:43.864889 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.864859    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865038 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.864894    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f40424dd-b515-4029-b062-02fbae312213-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865038 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.864929    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f40424dd-b515-4029-b062-02fbae312213-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865038 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.864962    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865038 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.865003    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.865059    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.865075    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnwf2\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-kube-api-access-hnwf2\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.865095    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.865253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.865115    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.965704 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965674    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.965704 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965708    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f40424dd-b515-4029-b062-02fbae312213-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.965935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965745    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f40424dd-b515-4029-b062-02fbae312213-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.965935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965780    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.965935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965829    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.965935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965900    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.965935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965919    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnwf2\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-kube-api-access-hnwf2\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.966249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965948    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.966249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.965969    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.966249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.966091    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.966508 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.966477    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f40424dd-b515-4029-b062-02fbae312213-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.966508 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.966500    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.966702 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.966537    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.966702 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.966575    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.968162 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.968135    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:43.968312 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:43.968297    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f40424dd-b515-4029-b062-02fbae312213-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:44.024031 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.023970    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:44.024031 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.023987    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnwf2\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-kube-api-access-hnwf2\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:44.050512 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.050484    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:44.219533 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.219506    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"]
Oct 30 21:51:44.220547 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:51:44.220521    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b174d9_930f_4553_b456_cf975d3612f8.slice/crio-473065c272a7d8a4b99c0d431f766ef128c93fcdd0ade214e793cbdc0f7501c8 WatchSource:0}: Error finding container 473065c272a7d8a4b99c0d431f766ef128c93fcdd0ade214e793cbdc0f7501c8: Status 404 returned error can't find the container with id 473065c272a7d8a4b99c0d431f766ef128c93fcdd0ade214e793cbdc0f7501c8
Oct 30 21:51:44.223056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.223022    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:44.223159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.223090    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:44.223159 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.223121    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:44.427498 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.427473    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"]
Oct 30 21:51:44.429495 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:51:44.429465    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40424dd_b515_4029_b062_02fbae312213.slice/crio-6bbb44066a72bca773a60f55825a03d4cab2692cc0aebc67409b8f74bd7e5207 WatchSource:0}: Error finding container 6bbb44066a72bca773a60f55825a03d4cab2692cc0aebc67409b8f74bd7e5207: Status 404 returned error can't find the container with id 6bbb44066a72bca773a60f55825a03d4cab2692cc0aebc67409b8f74bd7e5207
Oct 30 21:51:44.431910 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.431874    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:44.432018 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.431951    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:44.432018 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.431995    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:44.883532 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.883491    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" event={"ID":"f40424dd-b515-4029-b062-02fbae312213","Type":"ContainerStarted","Data":"e9c432a74870040f54a270481dd4e041a93941f747034aa8c3afe149e512f357"}
Oct 30 21:51:44.884003 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.883559    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" event={"ID":"f40424dd-b515-4029-b062-02fbae312213","Type":"ContainerStarted","Data":"6bbb44066a72bca773a60f55825a03d4cab2692cc0aebc67409b8f74bd7e5207"}
Oct 30 21:51:44.885426 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.885395    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" event={"ID":"a3b174d9-930f-4553-b456-cf975d3612f8","Type":"ContainerStarted","Data":"43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d"}
Oct 30 21:51:44.885515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:44.885438    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" event={"ID":"a3b174d9-930f-4553-b456-cf975d3612f8","Type":"ContainerStarted","Data":"473065c272a7d8a4b99c0d431f766ef128c93fcdd0ade214e793cbdc0f7501c8"}
Oct 30 21:51:45.005495 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.005440    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podStartSLOduration=2.005425991 podStartE2EDuration="2.005425991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:45.003037384 +0000 UTC m=+1053.280730858" watchObservedRunningTime="2026-10-30 21:51:45.005425991 +0000 UTC m=+1053.283119462"
Oct 30 21:51:45.061374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.051718    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:45.079374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.070305    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:51:45.079374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.070394    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:51:45.136875 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.136776    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podStartSLOduration=2.136763112 podStartE2EDuration="2.136763112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:45.13536846 +0000 UTC m=+1053.413061932" watchObservedRunningTime="2026-10-30 21:51:45.136763112 +0000 UTC m=+1053.414456644"
Oct 30 21:51:45.791375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.790730    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:45.795162 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.795130    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:51:45.795286 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:45.795180    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:51:46.058287 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:46.058197    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:51:46.059686 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:46.058849    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:51:46.792831 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:46.792788    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:51:46.793121 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:46.793085    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:51:47.055453 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:47.052679    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:51:47.055453 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:47.052737    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:51:47.801944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:47.801789    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:51:47.801944 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:47.801838    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:51:48.053771 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:48.053681    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:51:48.054015 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:48.053988    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:51:48.790767 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:48.790729    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:51:48.794557 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:48.790950    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:51:49.066432 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:49.060043    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:51:49.066432 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:49.060105    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:51:49.797165 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:49.797124    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:51:49.797493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:49.797459    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:51:50.051782 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:50.051690    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:51:50.052056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:50.052024    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:51:50.791224 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:50.791190    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:51:50.791704 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:50.791248    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:51:51.050829 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.050743    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:51:51.050829 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.050811    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:51:51.597573 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.597543    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"]
Oct 30 21:51:51.601559 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.601537    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.626557 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.626533    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zqcr\" (UniqueName: \"kubernetes.io/projected/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-kube-api-access-9zqcr\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.626683 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.626596    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-dshm\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.626683 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.626675    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-home\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.626794 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.626699    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tmp-dir\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.626794 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.626788    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tls-certs\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.626865 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.626834    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-model-cache\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.704877 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.704846    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-c3e2d8\"/\"kube-root-ca.crt\""
Oct 30 21:51:51.705062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.704901    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-c3e2d8\"/\"default-dockercfg-skv5v\""
Oct 30 21:51:51.705062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.704925    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-c3e2d8\"/\"distinct-w4-c3e2d8-kserve-self-signed-certs\""
Oct 30 21:51:51.705062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.704906    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-c3e2d8\"/\"openshift-service-ca.crt\""
Oct 30 21:51:51.707457 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.707435    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"]
Oct 30 21:51:51.728437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.728411    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tls-certs\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.728556 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.728479    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-model-cache\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.728624 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.728554    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zqcr\" (UniqueName: \"kubernetes.io/projected/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-kube-api-access-9zqcr\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.728681 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.728650    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-dshm\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.728741 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.728697    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-home\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.728741 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.728724    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tmp-dir\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.728969 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.728948    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-model-cache\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.729101 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.729081    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-home\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.729101 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.729095    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tmp-dir\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.730809 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.730789    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-dshm\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.730880 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.730864    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tls-certs\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.818928 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.818890    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:51.819431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.819216    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:51.820447 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.820425    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 21:51:51.830770 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.830751    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zqcr\" (UniqueName: \"kubernetes.io/projected/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-kube-api-access-9zqcr\") pod \"distinct-w4-c3e2d8-kserve-855c9694c6-dvslj\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:51.914329 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:51.914301    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:51:52.064913 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:52.064090    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:52.064913 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:52.064708    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:52.068120 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:52.068092    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 21:51:52.292313 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:52.290899    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"]
Oct 30 21:51:52.297244 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:51:52.297205    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb22cd2b0_4dcc_4791_9f65_cb9f34f9bb37.slice/crio-73985d5f347030054307c83c17eb82525f943d8406bb3d8608c1ce80a05b99b3 WatchSource:0}: Error finding container 73985d5f347030054307c83c17eb82525f943d8406bb3d8608c1ce80a05b99b3: Status 404 returned error can't find the container with id 73985d5f347030054307c83c17eb82525f943d8406bb3d8608c1ce80a05b99b3
Oct 30 21:51:52.966326 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:52.966291    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" event={"ID":"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37","Type":"ContainerStarted","Data":"de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263"}
Oct 30 21:51:52.966799 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:52.966339    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" event={"ID":"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37","Type":"ContainerStarted","Data":"73985d5f347030054307c83c17eb82525f943d8406bb3d8608c1ce80a05b99b3"}
Oct 30 21:51:52.980712 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:51:52.980674    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:51:53.084009 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:53.083955    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podStartSLOduration=2.083942837 podStartE2EDuration="2.083942837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:53.082642274 +0000 UTC m=+1061.360335748" watchObservedRunningTime="2026-10-30 21:51:53.083942837 +0000 UTC m=+1061.361636308"
Oct 30 21:51:53.915406 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:51:53.915380    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:52:01.915256 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:01.915214    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:52:01.927661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:01.927632    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:52:02.030840 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:02.030810    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:52:04.869935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:04.869901    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:52:22.981953 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:52:22.981869    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:52:27.796881 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.796843    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"]
Oct 30 21:52:27.800779 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.800757    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.854018 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.853995    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-c3e2d8-openshift-default-dockercfg-hqs2q\""
Oct 30 21:52:27.886624 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.886598    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.886743 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.886629    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.886743 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.886652    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2xsl\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-kube-api-access-t2xsl\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.886918 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.886891    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.886992 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.886974    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.887041 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.887013    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.887090 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.887051    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.887183 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.887164    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.887224 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.887189    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d09b71b-828c-4956-9d0b-d2da25d89b50-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.907095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.907067    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"]
Oct 30 21:52:27.988438 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988410    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988595 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988453    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988595 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988491    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988595 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988527    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988759 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988597    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988759 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988621    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d09b71b-828c-4956-9d0b-d2da25d89b50-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988759 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988672    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988759 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988695    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.988962 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988782    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2xsl\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-kube-api-access-t2xsl\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.989018 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.988999    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.989077 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.989043    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.989146 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.989119    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.989386 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.989334    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d09b71b-828c-4956-9d0b-d2da25d89b50-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.989468 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.989444    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.991125 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.991097    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:27.991229 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:27.991169    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:28.049034 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:28.048973    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:28.049619 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:28.049601    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2xsl\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-kube-api-access-t2xsl\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:28.113505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:28.113484    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:28.540496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:28.540470    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"]
Oct 30 21:52:28.542494 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:52:28.542464    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d09b71b_828c_4956_9d0b_d2da25d89b50.slice/crio-f3d2f361600cfde12937cb16e0db0b13097b7a278c0ab18af5ec05b777aed376 WatchSource:0}: Error finding container f3d2f361600cfde12937cb16e0db0b13097b7a278c0ab18af5ec05b777aed376: Status 404 returned error can't find the container with id f3d2f361600cfde12937cb16e0db0b13097b7a278c0ab18af5ec05b777aed376
Oct 30 21:52:28.544746 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:28.544706    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:28.544858 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:28.544788    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:28.544858 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:28.544836    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:29.194942 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:29.194905    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" event={"ID":"3d09b71b-828c-4956-9d0b-d2da25d89b50","Type":"ContainerStarted","Data":"be2de714aedc492fa564c8673274857383f7f99c2ddbeb0c1474910757f8c797"}
Oct 30 21:52:29.195440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:29.194970    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" event={"ID":"3d09b71b-828c-4956-9d0b-d2da25d89b50","Type":"ContainerStarted","Data":"f3d2f361600cfde12937cb16e0db0b13097b7a278c0ab18af5ec05b777aed376"}
Oct 30 21:52:29.369124 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:29.368884    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podStartSLOduration=2.368863483 podStartE2EDuration="2.368863483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:29.367819859 +0000 UTC m=+1097.645513331" watchObservedRunningTime="2026-10-30 21:52:29.368863483 +0000 UTC m=+1097.646556956"
Oct 30 21:52:30.114397 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.114339    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:30.115729 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.115703    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:52:30.115861 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.115749    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 21:52:30.793103 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.793068    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"]
Oct 30 21:52:30.796874 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.796852    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:30.871753 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.871714    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"]
Oct 30 21:52:30.911195 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.911163    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-557708\"/\"kube-root-ca.crt\""
Oct 30 21:52:30.911381 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.911334    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-557708\"/\"distinct-w6-557708-kserve-self-signed-certs\""
Oct 30 21:52:30.911381 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.911364    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-557708\"/\"openshift-service-ca.crt\""
Oct 30 21:52:30.911460 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.911389    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-557708\"/\"default-dockercfg-g4br6\""
Oct 30 21:52:30.940520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.932197    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7q88\" (UniqueName: \"kubernetes.io/projected/a9667903-bf26-4c66-b4cf-fecb659a03f2-kube-api-access-g7q88\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:30.940520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.938487    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-dshm\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:30.940520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.938539    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9667903-bf26-4c66-b4cf-fecb659a03f2-tls-certs\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:30.940520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.938686    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-tmp-dir\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:30.940520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.938949    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-model-cache\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:30.940520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:30.938992    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-home\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.039855 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.039820    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-model-cache\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.040112 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.040093    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-home\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.040271 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.040238    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-model-cache\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.040340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.040261    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7q88\" (UniqueName: \"kubernetes.io/projected/a9667903-bf26-4c66-b4cf-fecb659a03f2-kube-api-access-g7q88\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.040521 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.040451    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-home\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.040651 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.040634    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-dshm\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.040749 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.040736    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9667903-bf26-4c66-b4cf-fecb659a03f2-tls-certs\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.040910 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.040897    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-tmp-dir\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.041131 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.041114    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-tmp-dir\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.063235 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.063169    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-dshm\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.078383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.078233    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9667903-bf26-4c66-b4cf-fecb659a03f2-tls-certs\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.098748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.098721    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7q88\" (UniqueName: \"kubernetes.io/projected/a9667903-bf26-4c66-b4cf-fecb659a03f2-kube-api-access-g7q88\") pod \"distinct-w6-557708-kserve-bcc77c68c-f7px2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") " pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.109366 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.109329    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:31.114685 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.114656    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:52:31.114773 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.114718    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 21:52:31.634970 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:31.634935    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"]
Oct 30 21:52:31.636145 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:52:31.636098    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9667903_bf26_4c66_b4cf_fecb659a03f2.slice/crio-496fc99e7e765f49fda4b1e4671be874625e8c816aa6049a8f4b8e0981f01f8c WatchSource:0}: Error finding container 496fc99e7e765f49fda4b1e4671be874625e8c816aa6049a8f4b8e0981f01f8c: Status 404 returned error can't find the container with id 496fc99e7e765f49fda4b1e4671be874625e8c816aa6049a8f4b8e0981f01f8c
Oct 30 21:52:32.114161 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:32.114082    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:52:32.114161 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:32.114129    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 21:52:32.227857 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:32.227821    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" event={"ID":"a9667903-bf26-4c66-b4cf-fecb659a03f2","Type":"ContainerStarted","Data":"900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81"}
Oct 30 21:52:32.228050 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:32.227873    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" event={"ID":"a9667903-bf26-4c66-b4cf-fecb659a03f2","Type":"ContainerStarted","Data":"496fc99e7e765f49fda4b1e4671be874625e8c816aa6049a8f4b8e0981f01f8c"}
Oct 30 21:52:32.364424 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:32.361624    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podStartSLOduration=2.361606777 podStartE2EDuration="2.361606777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:32.360576665 +0000 UTC m=+1100.638270138" watchObservedRunningTime="2026-10-30 21:52:32.361606777 +0000 UTC m=+1100.639300250"
Oct 30 21:52:32.897739 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:32.897706    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:52:33.110327 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:33.110296    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:33.114641 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:33.114615    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:52:33.114956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:33.114672    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 21:52:34.114374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:34.114304    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:52:34.114374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:34.114373    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 21:52:35.114339 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:35.114298    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:52:35.114776 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:35.114368    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 21:52:36.153790 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:36.153757    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:36.154260 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:36.154195    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:36.154971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:36.154948    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 21:52:41.109736 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:41.109693    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:41.122941 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:41.122918    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:41.297031 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:41.296997    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:52:52.981297 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:52:52.981257    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:52:54.037149 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.034791    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"]
Oct 30 21:52:54.042039 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.042011    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.094283 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.094246    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"]
Oct 30 21:52:54.141067 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.141042    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bc009f\"/\"distinct-2-w3-bc009f-kserve-self-signed-certs\""
Oct 30 21:52:54.164960 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.164933    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-home\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.165087 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.164963    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj4bp\" (UniqueName: \"kubernetes.io/projected/4bd4292f-6a46-4aa4-9166-1637614c68f0-kube-api-access-gj4bp\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.165087 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.164996    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-dshm\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.165087 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.165069    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-model-cache\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.165205 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.165156    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bd4292f-6a46-4aa4-9166-1637614c68f0-tls-certs\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.165270 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.165231    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-tmp-dir\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.266476 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266441    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-dshm\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.266476 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266479    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-model-cache\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.266727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266608    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bd4292f-6a46-4aa4-9166-1637614c68f0-tls-certs\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.266727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266698    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-tmp-dir\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.266904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266876    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-home\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.266990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266914    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj4bp\" (UniqueName: \"kubernetes.io/projected/4bd4292f-6a46-4aa4-9166-1637614c68f0-kube-api-access-gj4bp\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.266990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266950    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-model-cache\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.267070 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.266998    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-tmp-dir\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.267198 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.267181    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-home\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.268856 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.268833    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-dshm\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.269011 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.268994    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bd4292f-6a46-4aa4-9166-1637614c68f0-tls-certs\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.324832 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.324778    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj4bp\" (UniqueName: \"kubernetes.io/projected/4bd4292f-6a46-4aa4-9166-1637614c68f0-kube-api-access-gj4bp\") pod \"distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") " pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.354618 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.354593    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:52:54.799813 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:54.799779    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"]
Oct 30 21:52:54.801245 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:52:54.801219    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bd4292f_6a46_4aa4_9166_1637614c68f0.slice/crio-68cb6feb063101649799cd99daa54250deb2ae32f7d848708cf50604a12ebf07 WatchSource:0}: Error finding container 68cb6feb063101649799cd99daa54250deb2ae32f7d848708cf50604a12ebf07: Status 404 returned error can't find the container with id 68cb6feb063101649799cd99daa54250deb2ae32f7d848708cf50604a12ebf07
Oct 30 21:52:55.363301 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:55.363251    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" event={"ID":"4bd4292f-6a46-4aa4-9166-1637614c68f0","Type":"ContainerStarted","Data":"703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b"}
Oct 30 21:52:55.363301 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:55.363299    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" event={"ID":"4bd4292f-6a46-4aa4-9166-1637614c68f0","Type":"ContainerStarted","Data":"68cb6feb063101649799cd99daa54250deb2ae32f7d848708cf50604a12ebf07"}
Oct 30 21:52:55.480360 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:55.480309    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podStartSLOduration=2.480297216 podStartE2EDuration="2.480297216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:55.479736571 +0000 UTC m=+1123.757430043" watchObservedRunningTime="2026-10-30 21:52:55.480297216 +0000 UTC m=+1123.757990686"
Oct 30 21:52:56.355712 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:52:56.355680    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:53:04.355160 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:04.355126    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:53:04.367769 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:04.367744    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:53:04.425217 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:04.425189    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:53:05.441963 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:05.441933    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:53:12.931616 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.931579    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"]
Oct 30 21:53:12.936491 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.936474    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986447 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986423    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hhp5\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-kube-api-access-9hhp5\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986558 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986456    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986558 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986485    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986632 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986606    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986711 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986691    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986746 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986738    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986782 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986767    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986814 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986796    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:12.986901 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:12.986884    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.087865 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.087832    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.087886    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.087920    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.087944    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088044 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.087979    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088271 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088059    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hhp5\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-kube-api-access-9hhp5\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088271 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088085    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088271 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088111    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088271 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088166    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088523 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088408    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088677 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088650    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088845 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088686    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088948 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088718    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.088948 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.088753    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.090570 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.090541    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.090928 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.090902    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.142606 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.142576    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"]
Oct 30 21:53:13.146676 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.146648    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hhp5\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-kube-api-access-9hhp5\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.147065 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.147043    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.249212 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.249132    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:13.676749 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.676724    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"]
Oct 30 21:53:13.678738 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:53:13.678713    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc08696_54ad_47eb_9f3c_07fe96105a8b.slice/crio-8c4b0a83a2bb4e2386b4beb6325d21cc882e39b2325bce5f1fa0175842123fde WatchSource:0}: Error finding container 8c4b0a83a2bb4e2386b4beb6325d21cc882e39b2325bce5f1fa0175842123fde: Status 404 returned error can't find the container with id 8c4b0a83a2bb4e2386b4beb6325d21cc882e39b2325bce5f1fa0175842123fde
Oct 30 21:53:13.681166 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.681134    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:13.681223 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.681198    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:13.681268 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:13.681230    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:14.487623 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:14.487580    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" event={"ID":"2fc08696-54ad-47eb-9f3c-07fe96105a8b","Type":"ContainerStarted","Data":"417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58"}
Oct 30 21:53:14.488050 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:14.487634    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" event={"ID":"2fc08696-54ad-47eb-9f3c-07fe96105a8b","Type":"ContainerStarted","Data":"8c4b0a83a2bb4e2386b4beb6325d21cc882e39b2325bce5f1fa0175842123fde"}
Oct 30 21:53:14.671800 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:14.671738    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podStartSLOduration=2.671723205 podStartE2EDuration="2.671723205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:14.668916886 +0000 UTC m=+1142.946610370" watchObservedRunningTime="2026-10-30 21:53:14.671723205 +0000 UTC m=+1142.949416737"
Oct 30 21:53:15.264377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:15.257776    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:15.269562 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:15.269537    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:53:15.269687 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:15.269583    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:53:16.262742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:16.262703    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:53:16.263322 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:16.263288    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:53:17.269082 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:17.269034    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:53:17.269686 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:17.269640    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:53:18.263725 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:18.263686    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:53:18.263991 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:18.263958    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:53:19.251800 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:19.251760    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:53:19.252424 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:19.252387    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:53:20.250407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:20.250375    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:53:20.250575 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:20.250423    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:53:21.254174 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:21.254144    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:21.254638 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:21.254617    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:21.255173 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:21.255154    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 21:53:22.982104 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:53:22.982066    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:53:43.429017 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.428976    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"]
Oct 30 21:53:43.433069 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.433047    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.536684 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.536647    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-c3e2d8\"/\"distinct-2-w4-c3e2d8-kserve-self-signed-certs\""
Oct 30 21:53:43.539221 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.539198    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"]
Oct 30 21:53:43.598409 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.598373    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tls-certs\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.598759 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.598733    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-dshm\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.598975 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.598957    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tmp-dir\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.627188 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.626421    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-model-cache\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.627188 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.626508    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-home\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.627188 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.626591    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5x45\" (UniqueName: \"kubernetes.io/projected/b887b392-ad21-4cfa-893e-ee93afcfc3d5-kube-api-access-q5x45\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.729435 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.729336    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-model-cache\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.729597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.729440    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-home\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.729597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.729514    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5x45\" (UniqueName: \"kubernetes.io/projected/b887b392-ad21-4cfa-893e-ee93afcfc3d5-kube-api-access-q5x45\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.729710 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.729613    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tls-certs\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.729710 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.729669    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-dshm\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.729806 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.729739    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tmp-dir\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.730283 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.730256    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tmp-dir\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.730587 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.730567    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-model-cache\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.730844 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.730825    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-home\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.734236 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.734207    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tls-certs\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.736589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.736560    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-dshm\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:43.788766 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:43.788743    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5x45\" (UniqueName: \"kubernetes.io/projected/b887b392-ad21-4cfa-893e-ee93afcfc3d5-kube-api-access-q5x45\") pod \"distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") " pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:44.045583 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:44.045508    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:44.419769 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:44.419738    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"]
Oct 30 21:53:44.421142 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:53:44.421113    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb887b392_ad21_4cfa_893e_ee93afcfc3d5.slice/crio-cf524374297e1624d1523865007e493815933aabfb2e3e479b08c72f29eecf18 WatchSource:0}: Error finding container cf524374297e1624d1523865007e493815933aabfb2e3e479b08c72f29eecf18: Status 404 returned error can't find the container with id cf524374297e1624d1523865007e493815933aabfb2e3e479b08c72f29eecf18
Oct 30 21:53:44.662259 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:44.662220    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" event={"ID":"b887b392-ad21-4cfa-893e-ee93afcfc3d5","Type":"ContainerStarted","Data":"b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679"}
Oct 30 21:53:44.662683 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:44.662268    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" event={"ID":"b887b392-ad21-4cfa-893e-ee93afcfc3d5","Type":"ContainerStarted","Data":"cf524374297e1624d1523865007e493815933aabfb2e3e479b08c72f29eecf18"}
Oct 30 21:53:44.780121 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:44.780006    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podStartSLOduration=1.779991024 podStartE2EDuration="1.779991024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:44.777778546 +0000 UTC m=+1173.055472019" watchObservedRunningTime="2026-10-30 21:53:44.779991024 +0000 UTC m=+1173.057684546"
Oct 30 21:53:45.045713 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:45.045629    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:46.223083 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:46.223052    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:53:52.982837 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:53:52.982752    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:53:54.046134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:54.046104    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:54.058792 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:54.058769    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:53:54.729581 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:53:54.729551    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:54:11.990258 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:11.990227    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:54:11.997302 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:11.997281    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:54:14.904290 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:14.904257    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"]
Oct 30 21:54:14.908032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:14.908015    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.018095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.018067    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"]
Oct 30 21:54:15.031779 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.031749    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.031920 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.031789    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.031985 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.031920    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.032043 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.032024    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd298109-a5d1-4b77-a486-fd147afff1d6-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.032092 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.032082    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.032226 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.032202    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.032268 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.032247    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.032412 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.032390    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.032466 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.032444    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqdfb\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-kube-api-access-rqdfb\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133297 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133229    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133297 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133272    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133533 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133314    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133533 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133381    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqdfb\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-kube-api-access-rqdfb\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133533 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133446    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133533 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133469    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133738 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133549    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133738 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133710    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd298109-a5d1-4b77-a486-fd147afff1d6-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.133848 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.133793    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.134442 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.134396    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.134442 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.134375    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.134639 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.134520    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.134800 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.134772    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.134946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.134929    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd298109-a5d1-4b77-a486-fd147afff1d6-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.136774 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.136747    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.139797 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.139772    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.192656 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.192588    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqdfb\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-kube-api-access-rqdfb\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.193181 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.193162    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.220578 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.220551    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:15.647170 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.647146    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"]
Oct 30 21:54:15.649024 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:54:15.648996    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd298109_a5d1_4b77_a486_fd147afff1d6.slice/crio-44b8322ec63e9ea5dbe16a10d7e67afc6b29c2b971b12fdee2741954a22fa096 WatchSource:0}: Error finding container 44b8322ec63e9ea5dbe16a10d7e67afc6b29c2b971b12fdee2741954a22fa096: Status 404 returned error can't find the container with id 44b8322ec63e9ea5dbe16a10d7e67afc6b29c2b971b12fdee2741954a22fa096
Oct 30 21:54:15.651336 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.651303    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:15.651428 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.651400    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:15.651470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.651431    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:15.847170 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.847137    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" event={"ID":"bd298109-a5d1-4b77-a486-fd147afff1d6","Type":"ContainerStarted","Data":"95f61289b121034e87c4a084d5c7a818b9bfaeb6aa69834c70b043871a141c46"}
Oct 30 21:54:15.847330 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.847186    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" event={"ID":"bd298109-a5d1-4b77-a486-fd147afff1d6","Type":"ContainerStarted","Data":"44b8322ec63e9ea5dbe16a10d7e67afc6b29c2b971b12fdee2741954a22fa096"}
Oct 30 21:54:15.970568 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:15.970470    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podStartSLOduration=1.970451063 podStartE2EDuration="1.970451063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:15.96844677 +0000 UTC m=+1204.246140244" watchObservedRunningTime="2026-10-30 21:54:15.970451063 +0000 UTC m=+1204.248144535"
Oct 30 21:54:16.221667 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:16.221579    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:16.222898 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:16.222868    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:54:16.223030 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:16.222927    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:54:17.220893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:17.220860    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:54:17.221306 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:17.220919    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:54:18.221370 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:18.221320    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:54:18.221839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:18.221384    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:54:19.221478 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:19.221440    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:54:19.221886 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:19.221493    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:54:20.221469 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:20.221431    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:54:20.221640 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:20.221488    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:54:21.221021 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:21.220986    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:54:21.221200 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:21.221045    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:54:22.221639 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:22.221604    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:54:22.222045 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:22.221660    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:54:22.983898 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:54:22.983857    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:54:23.225495 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:23.225466    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:23.225883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:23.225715    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:23.226346 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:23.226329    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 21:54:28.372750 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:28.372713    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:54:37.301008 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.300972    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"]
Oct 30 21:54:37.304864 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.304838    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.365286 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.365256    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"]
Oct 30 21:54:37.395835 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.395799    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-home\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.395988 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.395954    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tls-certs\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.396046 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.396028    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-model-cache\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.396103 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.396080    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz9gk\" (UniqueName: \"kubernetes.io/projected/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-kube-api-access-gz9gk\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.396214 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.396198    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-dshm\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.396276 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.396234    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tmp-dir\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.411297 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.411277    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c79cb5\"/\"distinct-2-w2-c79cb5-kserve-self-signed-certs\""
Oct 30 21:54:37.503377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.499004    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tls-certs\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.503377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.499066    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-model-cache\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.503377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.499115    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz9gk\" (UniqueName: \"kubernetes.io/projected/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-kube-api-access-gz9gk\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.503377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.499229    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-dshm\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.503377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.499261    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tmp-dir\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.503377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.499381    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-home\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.503377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.499841    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-home\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.514519 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.509274    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-model-cache\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.514519 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.510835    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tmp-dir\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.517946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.517919    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-dshm\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.526254 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.525648    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tls-certs\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.579822 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.579754    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz9gk\" (UniqueName: \"kubernetes.io/projected/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-kube-api-access-gz9gk\") pod \"distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") " pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:37.617665 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:37.617590    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:38.021542 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:38.021505    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"]
Oct 30 21:54:38.022755 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:54:38.022721    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod709f5224_60a7_4eb5_b8ea_3e08ed9c2c0a.slice/crio-50e3805be5b2a5cb404f0a9b6bc126f392dd2122a1efc8a8909c52163ed66f0e WatchSource:0}: Error finding container 50e3805be5b2a5cb404f0a9b6bc126f392dd2122a1efc8a8909c52163ed66f0e: Status 404 returned error can't find the container with id 50e3805be5b2a5cb404f0a9b6bc126f392dd2122a1efc8a8909c52163ed66f0e
Oct 30 21:54:38.158573 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:38.158501    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" event={"ID":"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a","Type":"ContainerStarted","Data":"5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878"}
Oct 30 21:54:38.158573 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:38.158553    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" event={"ID":"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a","Type":"ContainerStarted","Data":"50e3805be5b2a5cb404f0a9b6bc126f392dd2122a1efc8a8909c52163ed66f0e"}
Oct 30 21:54:38.277659 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:38.277557    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podStartSLOduration=1.27754282 podStartE2EDuration="1.27754282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:38.274993178 +0000 UTC m=+1226.552686650" watchObservedRunningTime="2026-10-30 21:54:38.27754282 +0000 UTC m=+1226.555236344"
Oct 30 21:54:38.618095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:38.618070    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:45.226432 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.224793    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"]
Oct 30 21:54:45.232690 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.232537    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293225 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293192    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b089a3-2fc6-4068-ba97-21bf694af449-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293234    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4kp2\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-kube-api-access-q4kp2\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293366    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293483 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293410    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293483 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293428    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293551 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293527    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293620 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293595    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293763    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.293860 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.293827    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b089a3-2fc6-4068-ba97-21bf694af449-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.341902 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.341866    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"]
Oct 30 21:54:45.395414 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395374    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395588 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395450    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b089a3-2fc6-4068-ba97-21bf694af449-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395588 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395492    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b089a3-2fc6-4068-ba97-21bf694af449-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395588 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395528    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4kp2\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-kube-api-access-q4kp2\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395619    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395668    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395863 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395765    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395863 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395823    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.395965 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.395889    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.396098 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.396036    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.396098 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.396057    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.396289 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.396233    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.396289 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.396273    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.396522 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.396496    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b089a3-2fc6-4068-ba97-21bf694af449-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.397857 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.397836    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.398074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.398054    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b089a3-2fc6-4068-ba97-21bf694af449-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.455527 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.455494    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4kp2\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-kube-api-access-q4kp2\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.455666 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.455570    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.546567 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.546496    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:45.971835 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.971811    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"]
Oct 30 21:54:45.973902 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:54:45.973875    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b089a3_2fc6_4068_ba97_21bf694af449.slice/crio-b3f55e81a6240fd47a6cdf299f537f072defc8b8c1f379f4b622bcc3e4fb87bf WatchSource:0}: Error finding container b3f55e81a6240fd47a6cdf299f537f072defc8b8c1f379f4b622bcc3e4fb87bf: Status 404 returned error can't find the container with id b3f55e81a6240fd47a6cdf299f537f072defc8b8c1f379f4b622bcc3e4fb87bf
Oct 30 21:54:45.976737 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.976709    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:45.976801 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.976778    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:45.976838 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:45.976808    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:46.209281 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:46.209243    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" event={"ID":"23b089a3-2fc6-4068-ba97-21bf694af449","Type":"ContainerStarted","Data":"15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da"}
Oct 30 21:54:46.209499 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:46.209302    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" event={"ID":"23b089a3-2fc6-4068-ba97-21bf694af449","Type":"ContainerStarted","Data":"b3f55e81a6240fd47a6cdf299f537f072defc8b8c1f379f4b622bcc3e4fb87bf"}
Oct 30 21:54:46.330910 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:46.330795    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podStartSLOduration=1.330775292 podStartE2EDuration="1.330775292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:46.329479586 +0000 UTC m=+1234.607173057" watchObservedRunningTime="2026-10-30 21:54:46.330775292 +0000 UTC m=+1234.608468765"
Oct 30 21:54:46.556395 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:46.548918    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:47.076907 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.076874    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"]
Oct 30 21:54:47.080781 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.080763    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.176724 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.176688    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"]
Oct 30 21:54:47.208375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.208325    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-557708\"/\"distinct-2-w6-557708-kserve-self-signed-certs\""
Oct 30 21:54:47.259274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.237024    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tls-certs\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.259274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.237072    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-dshm\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.259274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.237135    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-home\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.259274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.237235    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-model-cache\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.259274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.237257    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tmp-dir\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.259274 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.237292    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rhcd\" (UniqueName: \"kubernetes.io/projected/9b80c40d-dd47-403d-ae15-2ef2c658a53f-kube-api-access-8rhcd\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.343222 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.343132    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-model-cache\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.343222 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.343178    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tmp-dir\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.343771 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.343245    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rhcd\" (UniqueName: \"kubernetes.io/projected/9b80c40d-dd47-403d-ae15-2ef2c658a53f-kube-api-access-8rhcd\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.362376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.344526    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-model-cache\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.362376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.344864    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tmp-dir\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.362376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.343924    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tls-certs\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.362376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.346722    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-dshm\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.362376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.346895    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-home\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.362376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.347894    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-home\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.367368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.366209    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tls-certs\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.376372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.371644    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-dshm\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.406881 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.406851    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rhcd\" (UniqueName: \"kubernetes.io/projected/9b80c40d-dd47-403d-ae15-2ef2c658a53f-kube-api-access-8rhcd\") pod \"distinct-2-w6-557708-kserve-5ff7d98644-9g4ph\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") " pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:47.604441 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.602703    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:54:47.604441 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.602766    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:54:47.618416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.618382    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:47.630609 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.630587    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:47.695006 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:47.694971    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:48.107168 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.105304    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"]
Oct 30 21:54:48.110992 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:54:48.110961    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b80c40d_dd47_403d_ae15_2ef2c658a53f.slice/crio-73df2f855ca703431f57e59b2ac4ddd73d157b454f29baca7cc591c14b32e920 WatchSource:0}: Error finding container 73df2f855ca703431f57e59b2ac4ddd73d157b454f29baca7cc591c14b32e920: Status 404 returned error can't find the container with id 73df2f855ca703431f57e59b2ac4ddd73d157b454f29baca7cc591c14b32e920
Oct 30 21:54:48.221700 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.221665    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" event={"ID":"9b80c40d-dd47-403d-ae15-2ef2c658a53f","Type":"ContainerStarted","Data":"e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb"}
Oct 30 21:54:48.221845 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.221714    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" event={"ID":"9b80c40d-dd47-403d-ae15-2ef2c658a53f","Type":"ContainerStarted","Data":"73df2f855ca703431f57e59b2ac4ddd73d157b454f29baca7cc591c14b32e920"}
Oct 30 21:54:48.239216 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.234216    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:54:48.370452 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.370344    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podStartSLOduration=1.3703294449999999 podStartE2EDuration="1.370329445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:48.368012376 +0000 UTC m=+1236.645705847" watchObservedRunningTime="2026-10-30 21:54:48.370329445 +0000 UTC m=+1236.648022917"
Oct 30 21:54:48.562451 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.555735    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:54:48.562451 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.555818    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:54:48.695934 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:48.695854    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:49.549739 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:49.549702    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:54:49.550336 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:49.550301    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:54:50.553372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:50.552718    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:54:50.553372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:50.552777    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:54:51.568377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:51.563777    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:54:51.568377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:51.563852    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:54:52.547434 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:52.547389    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:54:52.547632 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:52.547436    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:54:52.982569 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:54:52.982530    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:54:53.551612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:53.551577    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:53.551970 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:53.551954    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:53.552654 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:53.552636    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 21:54:57.696001 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:57.695964    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:57.708791 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:57.708764    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:54:58.360848 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:54:58.360811    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:55:00.231804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.231768    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"]
Oct 30 21:55:00.236631 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.236607    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.370481 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.370436    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.370756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.370729    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.370925 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.370905    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.371047 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.371032    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.371153 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.371139    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25dd90d4-46db-4987-8bf5-4ef74c025021-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.371296 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.371281    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w98vc\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-kube-api-access-w98vc\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.371538 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.371512    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.371854 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.371818    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.372063 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.372039    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.398379 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.373188    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"]
Oct 30 21:55:00.560217 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560119    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560217 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560203    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560507 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560295    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560507 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560333    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560507 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560391    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560507 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560421    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560507 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560453    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25dd90d4-46db-4987-8bf5-4ef74c025021-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560784 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560515    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w98vc\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-kube-api-access-w98vc\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.560784 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.560622    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.568020 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.567986    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.593150 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.586691    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.593150 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.587643    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.593150 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.588207    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.593150 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.588887    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25dd90d4-46db-4987-8bf5-4ef74c025021-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.593150 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.590426    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.596830 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.596803    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.677384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.677327    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.702374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.678582    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w98vc\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-kube-api-access-w98vc\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:00.852416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:00.852328    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:01.741123 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:01.741070    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"]
Oct 30 21:55:01.833832 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:01.813901    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:01.833832 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:01.813997    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:01.833832 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:01.814044    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:01.885657 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:01.885550    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.055946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.054772    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.055946 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.055147    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.205374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.199446    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.618563 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.618512    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.619846 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.619814    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.694421 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.678906    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:55:02.694945 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.694890    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.735963 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.733446    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.740069 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.738575    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" event={"ID":"25dd90d4-46db-4987-8bf5-4ef74c025021","Type":"ContainerStarted","Data":"0cacb3319df5c254ff37f9e7f82b44054d71ad0025df5a2d1347a9d15101f956"}
Oct 30 21:55:02.740069 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.738649    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" event={"ID":"25dd90d4-46db-4987-8bf5-4ef74c025021","Type":"ContainerStarted","Data":"4b886fd4b6edd86a78f66c3cc88abc4760faa1b1e50a6433c100471f5410fbc7"}
Oct 30 21:55:02.853486 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.853451    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:02.873642 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.873517    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podStartSLOduration=2.87349305 podStartE2EDuration="2.87349305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:02.861763629 +0000 UTC m=+1251.139457102" watchObservedRunningTime="2026-10-30 21:55:02.87349305 +0000 UTC m=+1251.151186521"
Oct 30 21:55:02.946517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.946462    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Oct 30 21:55:02.947102 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.947059    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:55:02.947683 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:02.947661    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:55:03.022493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.022440    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.057895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.057848    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:55:03.060120 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.059260    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:55:03.203392 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.201663    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.298377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.217474    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:55:03.349375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.305982    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.349375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.313204    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.433376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.404033    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:55:03.620488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.620416    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.622597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.622573    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:55:03.706496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.706439    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:55:03.718547 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.718511    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:55:03.744872 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.744816    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.776166 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.769511    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:55:03.815967 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.800143    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.815967 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.809436    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:55:03.819456 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.819323    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:03.819456 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.819397    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.839060 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.839029    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:55:03.867737 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.867702    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:55:03.879519 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:03.879418    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:55:04.027489 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.027442    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:04.170273 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.027750    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:04.171038 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.068477    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:55:04.236381 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.229375    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:55:04.291814 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.291777    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:55:04.309515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.309465    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:04.311024 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.311001    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:55:04.407391 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.407327    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:04.410121 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.410095    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:55:04.410489 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.410473    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:55:04.412524 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.412492    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:04.423414 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.416769    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:55:04.423414 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.419270    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:55:04.423414 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.419331    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:55:04.434250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.433297    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:55:04.434250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.433710    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:55:04.458374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.454982    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:55:04.862426 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.856745    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:55:04.862426 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:04.856806    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:55:05.858838 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:05.856344    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:55:05.858838 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:05.856424    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:55:06.865373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:06.864720    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:55:06.865373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:06.864781    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:55:07.854116 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:07.854085    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:55:07.854428 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:07.854399    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:55:08.869417 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:08.865581    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:55:08.869417 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:08.865676    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:55:09.853158 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:09.853127    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:55:09.853373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:09.853187    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:55:10.857284 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:10.857257    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:10.857749 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:10.857728    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:10.858439 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:10.858423    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 21:55:23.020376 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:55:23.020272    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:55:29.582466 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.582426    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"]
Oct 30 21:55:29.586441 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.586424    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.619641 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.619611    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.619761 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.619654    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwqk5\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-kube-api-access-zwqk5\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.619761 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.619747    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.619842 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.619782    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.619881 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.619852    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.619978 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.619956    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.620019 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.620002    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.620096 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.620081    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.620168 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.620156    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.683024 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.682990    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"]
Oct 30 21:55:29.721400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721343    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721586 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721406    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721586 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721441    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwqk5\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-kube-api-access-zwqk5\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721586 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721480    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721586 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721507    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721586 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721559    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721847 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721660    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721847 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721711    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.721847 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721761    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.722022 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.721992    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.722190 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.722112    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.722190 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.722171    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.722338 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.722156    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.722492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.722468    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.723914 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.723893    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.724076 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.724060    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.780316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.780286    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwqk5\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-kube-api-access-zwqk5\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.780432 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.780341    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:29.899702 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:29.899676    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:30.215658 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.215585    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"]
Oct 30 21:55:30.220505 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.220483    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.314848 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.314810    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"]
Oct 30 21:55:30.330932 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.330889    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d6a011-281b-4179-bf8d-4954ecb3df4d-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331110 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331023    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331110 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331046    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331245 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331157    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbvgd\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-kube-api-access-mbvgd\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331245 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331196    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331245 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331216    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331282    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331326    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.331448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.331371    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.377875 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.377839    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"]
Oct 30 21:55:30.378679 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:55:30.378649    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25b0f19_8746_49b5_a2a0_d9ad45116d6a.slice/crio-de8f0a1bd158e4b38510505eb8bbcfbad5fa7321643d6fd6338e7e07db2f02ca WatchSource:0}: Error finding container de8f0a1bd158e4b38510505eb8bbcfbad5fa7321643d6fd6338e7e07db2f02ca: Status 404 returned error can't find the container with id de8f0a1bd158e4b38510505eb8bbcfbad5fa7321643d6fd6338e7e07db2f02ca
Oct 30 21:55:30.380999 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.380966    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:30.381074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.381031    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:30.381074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.381059    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:30.432516 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.432491    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.432646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.432525    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.432646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.432555    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.432646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.432614    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.432819 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.432663    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.432876 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.432819    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d6a011-281b-4179-bf8d-4954ecb3df4d-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.432942 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.432914    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.433023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.433004    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.433076 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.433048    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.433076 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.433042    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.433176 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.433128    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.433228 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.433189    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbvgd\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-kube-api-access-mbvgd\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.433301 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.433279    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.433624 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.433591    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d6a011-281b-4179-bf8d-4954ecb3df4d-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.435615 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.435591    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.435922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.435901    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.493201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.493129    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.493724 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.493705    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbvgd\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-kube-api-access-mbvgd\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.543088 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.543057    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:30.977495 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:55:30.977463    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d6a011_281b_4179_bf8d_4954ecb3df4d.slice/crio-ea5e9628bb61978372ad545b7cb483e6c84152075836c1bbe08d1c5bb8d39a76 WatchSource:0}: Error finding container ea5e9628bb61978372ad545b7cb483e6c84152075836c1bbe08d1c5bb8d39a76: Status 404 returned error can't find the container with id ea5e9628bb61978372ad545b7cb483e6c84152075836c1bbe08d1c5bb8d39a76
Oct 30 21:55:30.978882 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.978857    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"]
Oct 30 21:55:30.980240 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.980203    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:30.980343 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.980272    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:30.980343 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:30.980299    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:31.049417 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:31.049384    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" event={"ID":"05d6a011-281b-4179-bf8d-4954ecb3df4d","Type":"ContainerStarted","Data":"ea5e9628bb61978372ad545b7cb483e6c84152075836c1bbe08d1c5bb8d39a76"}
Oct 30 21:55:31.051280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:31.051254    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" event={"ID":"b25b0f19-8746-49b5-a2a0-d9ad45116d6a","Type":"ContainerStarted","Data":"3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa"}
Oct 30 21:55:31.051419 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:31.051305    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" event={"ID":"b25b0f19-8746-49b5-a2a0-d9ad45116d6a","Type":"ContainerStarted","Data":"de8f0a1bd158e4b38510505eb8bbcfbad5fa7321643d6fd6338e7e07db2f02ca"}
Oct 30 21:55:31.176472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:31.176413    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podStartSLOduration=2.176397204 podStartE2EDuration="2.176397204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:31.172555994 +0000 UTC m=+1279.450249466" watchObservedRunningTime="2026-10-30 21:55:31.176397204 +0000 UTC m=+1279.454090675"
Oct 30 21:55:31.900918 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:31.900887    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:31.902101 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:31.902078    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:55:31.902170 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:31.902126    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:55:32.075375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:32.074713    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" event={"ID":"05d6a011-281b-4179-bf8d-4954ecb3df4d","Type":"ContainerStarted","Data":"c51f59f1aaad813a1bab35858caf2c678002c18ede6ae099bce13a9f10d8c8a2"}
Oct 30 21:55:32.199388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:32.199258    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podStartSLOduration=2.199241183 podStartE2EDuration="2.199241183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:32.199227892 +0000 UTC m=+1280.476921365" watchObservedRunningTime="2026-10-30 21:55:32.199241183 +0000 UTC m=+1280.476934656"
Oct 30 21:55:32.547433 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:32.544637    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:32.575377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:32.562583    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.575377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:32.562644    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:55:32.903822 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:32.903777    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.904129 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:32.903845    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:55:33.552376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:33.551764    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.552376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:33.551835    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:55:33.899996 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:33.899954    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.900174 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:33.900018    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:55:34.547736 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:34.547697    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.548001 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:34.547973    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:55:34.901394 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:34.900328    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.901394 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:34.900405    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:55:35.560748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:35.560710    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:55:35.561032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:35.561005    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:55:35.900742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:35.900707    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:55:35.901019 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:35.900989    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:55:36.555376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:36.554833    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.555376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:36.554909    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:55:36.900463 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:36.900431    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.900648 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:36.900478    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:55:37.543933 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:37.543901    5561 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.544101 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:37.543951    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:55:37.645130 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:37.645092    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:55:38.254910 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:38.254880    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:38.256655 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:38.255570    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:38.260442 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:38.260415    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 21:55:39.171457 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:39.169419    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:39.171457 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:39.170794    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:39.176628 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:39.176568    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 21:55:40.947043 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:40.946999    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.235431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.224861    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.235431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.225629    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.394651 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.360194    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.394651 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.376577    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.409490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.409451    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.426454 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.426411    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.447399 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.447339    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.481372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.480575    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.493518 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.493443    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.884658 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.884144    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:41.884658 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:41.884221    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:52.982087 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:55:52.982043    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:55:59.517789 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.517756    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"]
Oct 30 21:55:59.521662 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.521638    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610253 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610223    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610430 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610284    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610481 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610432    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610531 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610493    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610572 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610531    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610572 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610552    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610710 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610618    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lnwc\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-kube-api-access-5lnwc\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610710 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610669    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.610710 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.610684    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.630460 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.630430    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"]
Oct 30 21:55:59.712309 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712280    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.712517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712316    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.712517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712416    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.712517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712485    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.712697 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712575    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.712697 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712667    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.712697 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712692    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.713228 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.713199    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.713402 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712782    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.713402 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712978    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.713402 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.713051    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.713402 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.712721    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.713402 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.713291    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lnwc\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-kube-api-access-5lnwc\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.713733 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.713428    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.715003 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.714976    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.715135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.715099    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.775272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.775204    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.775272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.775207    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lnwc\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-kube-api-access-5lnwc\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:55:59.835017 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:55:59.834989    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:56:00.109429 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.109337    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"]
Oct 30 21:56:00.114286 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.114263    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.180224 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.180156    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"]
Oct 30 21:56:00.219650 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.219611    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.219811 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.219654    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.219972 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.219946    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxrdt\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-kube-api-access-gxrdt\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.220071 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.219990    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.220071 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.220016    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.220188 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.220172    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.220243 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.220227    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.220440 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.220415    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.220517 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.220470    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.278166 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.278143    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"]
Oct 30 21:56:00.278278 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:56:00.278256    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1afd3bc_5ed8_4475_8c83_1c5a2cbaabbd.slice/crio-749b9f18af439aae8caa830480ef17a3eb4ee9466674dc8e72ed0988c4421d21 WatchSource:0}: Error finding container 749b9f18af439aae8caa830480ef17a3eb4ee9466674dc8e72ed0988c4421d21: Status 404 returned error can't find the container with id 749b9f18af439aae8caa830480ef17a3eb4ee9466674dc8e72ed0988c4421d21
Oct 30 21:56:00.280651 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.280619    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:00.280722 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.280684    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:00.280722 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.280712    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:00.321493 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321469    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxrdt\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-kube-api-access-gxrdt\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321596 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321501    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321596 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321520    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321596 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321563    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321596    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321646    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321684    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321770    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.321810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321802    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.322094 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.321939    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.322262 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.322132    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.322431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.322407    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.322776 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.322757    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.323113 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.323088    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.323920 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.323899    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.325841 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.325821    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.381243 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.381217    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxrdt\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-kube-api-access-gxrdt\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.381848 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.381821    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.432594 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.432562    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:00.508269 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.507407    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" event={"ID":"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd","Type":"ContainerStarted","Data":"7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c"}
Oct 30 21:56:00.508269 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.507539    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" event={"ID":"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd","Type":"ContainerStarted","Data":"749b9f18af439aae8caa830480ef17a3eb4ee9466674dc8e72ed0988c4421d21"}
Oct 30 21:56:00.639762 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.639664    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podStartSLOduration=1.639650097 podStartE2EDuration="1.639650097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:00.638167157 +0000 UTC m=+1308.915860630" watchObservedRunningTime="2026-10-30 21:56:00.639650097 +0000 UTC m=+1308.917343567"
Oct 30 21:56:00.712971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.712939    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"]
Oct 30 21:56:00.718108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.718084    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.779886 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.779854    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"]
Oct 30 21:56:00.832904 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.832871    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a84d78-0397-4563-8b25-0d979527427b-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833081 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.832908    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833081 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.833052    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hdf6\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-kube-api-access-6hdf6\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833081 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.833080    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833258 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.833143    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a84d78-0397-4563-8b25-0d979527427b-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833258 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.833198    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833331 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.833257    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833331 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.833327    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.833496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.833479    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.835337 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.835315    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:56:00.850338 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.844955    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"]
Oct 30 21:56:00.851822 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:56:00.851790    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2544bf5_c6ac_4b3a_91f7_4de1f5fd8137.slice/crio-230f214f788d45a8e2be73784cdf23c670d47b405c63674547751a5ec3f5538a WatchSource:0}: Error finding container 230f214f788d45a8e2be73784cdf23c670d47b405c63674547751a5ec3f5538a: Status 404 returned error can't find the container with id 230f214f788d45a8e2be73784cdf23c670d47b405c63674547751a5ec3f5538a
Oct 30 21:56:00.857703 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.856634    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:00.857990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.857959    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:00.858134 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.858116    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:00.934483 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934390    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a84d78-0397-4563-8b25-0d979527427b-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934483 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934431    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934656 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934489    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934656 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934602    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934781 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934687    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934840 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934802    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a84d78-0397-4563-8b25-0d979527427b-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934840 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934829    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934855    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934923    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hdf6\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-kube-api-access-6hdf6\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.934952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934948    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.935107 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.934950    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.935671 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.935541    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a84d78-0397-4563-8b25-0d979527427b-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.935671 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.935638    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.935798 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.935762    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.937054 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.937033    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a84d78-0397-4563-8b25-0d979527427b-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.937208 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.937188    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.994051 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.994023    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hdf6\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-kube-api-access-6hdf6\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:00.994197 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:00.994099    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:01.065388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.062339    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:01.488513 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.488481    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"]
Oct 30 21:56:01.491778 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:56:01.491744    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a84d78_0397_4563_8b25_0d979527427b.slice/crio-c3f88d42cc145dde5277cce67bc128fe4a6034eb6bb971f985f2f5826f91a042 WatchSource:0}: Error finding container c3f88d42cc145dde5277cce67bc128fe4a6034eb6bb971f985f2f5826f91a042: Status 404 returned error can't find the container with id c3f88d42cc145dde5277cce67bc128fe4a6034eb6bb971f985f2f5826f91a042
Oct 30 21:56:01.494760 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.494722    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:01.494839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.494796    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:01.494839 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.494825    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:01.527042 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.525716    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" event={"ID":"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137","Type":"ContainerStarted","Data":"d7ab9b793531c68951239f77192513aab255871e34a33bc6dc5b6ac925b4908f"}
Oct 30 21:56:01.527042 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.525770    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" event={"ID":"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137","Type":"ContainerStarted","Data":"230f214f788d45a8e2be73784cdf23c670d47b405c63674547751a5ec3f5538a"}
Oct 30 21:56:01.528187 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.528053    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" event={"ID":"b5a84d78-0397-4563-8b25-0d979527427b","Type":"ContainerStarted","Data":"c3f88d42cc145dde5277cce67bc128fe4a6034eb6bb971f985f2f5826f91a042"}
Oct 30 21:56:01.726175 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.726079    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podStartSLOduration=1.726058598 podStartE2EDuration="1.726058598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:01.722519471 +0000 UTC m=+1310.000212944" watchObservedRunningTime="2026-10-30 21:56:01.726058598 +0000 UTC m=+1310.003752069"
Oct 30 21:56:01.836718 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.836632    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:56:01.836718 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:01.836686    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:56:02.434117 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:02.434085    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:02.436537 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:02.436261    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:56:02.436537 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:02.436314    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:56:02.542882 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:02.542850    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" event={"ID":"b5a84d78-0397-4563-8b25-0d979527427b","Type":"ContainerStarted","Data":"20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53"}
Oct 30 21:56:02.836247 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:02.836163    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:56:02.836247 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:02.836222    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:56:03.068455 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:03.068418    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:03.069129 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:03.069097    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:56:03.069250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:03.069156    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:56:03.434103 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:03.434064    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:56:03.434406 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:03.434369    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:56:03.835757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:03.835677    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:56:03.835757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:03.835734    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:56:04.064374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:04.063735    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:56:04.064374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:04.063797    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:56:04.439317 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:04.434277    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:56:04.439317 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:04.434333    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:56:04.835870 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:04.835790    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:56:04.835870 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:04.835844    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:56:05.065788 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:05.065750    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:56:05.066412 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:05.066377    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:56:05.434926 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:05.434212    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:56:05.434926 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:05.434268    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:56:05.836309 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:05.836216    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:56:05.836309 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:05.836293    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:56:06.077141 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:06.076702    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:56:06.077141 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:06.076764    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:56:06.435252 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:06.435130    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:56:06.435252 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:06.435228    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:56:06.839428 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:06.838879    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:56:06.839428 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:06.838965    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:56:07.066377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:07.063241    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:56:07.066377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:07.063313    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:56:07.434049 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:07.434011    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:56:07.434472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:07.434079    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:56:08.058404 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.052455    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:56:08.058404 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.052885    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:56:08.058404 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.054324    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 21:56:08.063096 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.063064    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:56:08.063269 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.063135    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:56:08.277165 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.277099    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podStartSLOduration=8.277079288 podStartE2EDuration="8.277079288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:02.72938815 +0000 UTC m=+1311.007081625" watchObservedRunningTime="2026-10-30 21:56:08.277079288 +0000 UTC m=+1316.554772760"
Oct 30 21:56:08.670523 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.670489    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:08.671797 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.671780    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:08.672452 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:08.672434    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 21:56:09.507587 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:09.507539    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:09.507961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:09.507936    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:09.509865 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:09.509837    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 21:56:17.188420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.185392    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.369375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.351504    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.405374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.404968    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.421384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.417448    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:56:17.421384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.418216    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.439423 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.432292    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:56:17.478482 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.478433    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:56:17.483463 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.483425    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.505735 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.484404    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.849980 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:17.849889    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:56:18.259375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.239434    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.259375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.240663    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:56:18.286375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.266481    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:56:18.464387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.382495    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:56:18.464387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.382952    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:56:18.464387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.436523    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.464387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.436879    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.464387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.437695    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:56:18.464387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.441502    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:56:18.582574 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.542488    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:56:18.706941 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.706899    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:56:18.750376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.726027    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:56:18.750376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.726256    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:56:18.760647 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.625747    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.777276 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.777244    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:56:18.822000 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.626686    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:56:18.851435 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.684935    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.878373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.875473    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.972378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.888261    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:56:18.972378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.888421    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:56:18.972378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.888506    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:56:18.972378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.952136    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.972378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:18.959718    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": context deadline exceeded"
Oct 30 21:56:19.246515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.246458    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:19.326379 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.305429    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:19.326379 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.317330    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:56:19.391525 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.391484    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.465503 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.465455    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.475085 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.475040    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:19.482683 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.482640    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:19.490374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.486657    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:19.809370 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.787003    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.821621 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.821567    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.984740 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.983700    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.984740 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:19.984277    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:20.004297 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.004241    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:20.248291 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.248248    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:56:20.452380 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.423829    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.156939 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.465002    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.157396 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.492803    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:56:21.162364 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.493454    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.162740 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.493975    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:56:21.166265 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.493997    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.168056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.822991    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.168387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.866084    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:21.168690 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.987612    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.168978 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.988981    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": context deadline exceeded"
Oct 30 21:56:21.169230 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:20.989428    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.196948 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.196921    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:56:21.269371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.267550    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:56:21.327488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.327455    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:56:21.335538 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.335515    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:56:21.338959 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.338943    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:56:21.339609 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.339586    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:56:21.340974 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.340956    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:56:21.341572 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.341339    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:56:21.341686 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.341670    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:56:21.359590 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.359560    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:56:21.389700 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.389670    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:56:21.404559 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:21.404526    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:56:22.983711 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:56:22.983673    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:56:44.902645 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:44.902613    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"]
Oct 30 21:56:44.907908 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:44.907888    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.014699 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.014663    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"]
Oct 30 21:56:45.096198 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096164    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096392 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096267    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096392 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096367    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84239b3b-2157-4921-aacd-ed21bc708701-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096501 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096433    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096560 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096532    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg7sl\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-kube-api-access-pg7sl\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096648 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096629    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096728 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096712    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096786 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096769    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.096922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.096909    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84239b3b-2157-4921-aacd-ed21bc708701-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.198727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.198640    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84239b3b-2157-4921-aacd-ed21bc708701-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.198727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.198700    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.198956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.198738    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg7sl\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-kube-api-access-pg7sl\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.198956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.198799    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.198956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.198851    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.198956 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.198900    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.199193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.199032    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84239b3b-2157-4921-aacd-ed21bc708701-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.199193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.199091    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.199193 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.199162    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.199693 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.199656    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.199820 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.199711    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.200025 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.199988    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.200190 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.200171    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84239b3b-2157-4921-aacd-ed21bc708701-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.200333 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.200310    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.201299 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.201280    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.201637 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.201610    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84239b3b-2157-4921-aacd-ed21bc708701-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.220739 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.220704    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"]
Oct 30 21:56:45.226191 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.226168    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.262136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.262102    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg7sl\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-kube-api-access-pg7sl\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.264858 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.264826    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.287224 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.287188    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"]
Oct 30 21:56:45.402453 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.402415    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.402634 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.402540    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.402702 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.402640    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.402757 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.402737    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrstj\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-kube-api-access-xrstj\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.402829 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.402812    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.402893 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.402866    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.402948 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.402910    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.403153 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.403133    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ff8aad-679b-4686-9b94-fb4441696b67-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.403207 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.403175    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.504678 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504596    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ff8aad-679b-4686-9b94-fb4441696b67-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.504678 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504640    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.504894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504692    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.504894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504728    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.504894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504790    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.504894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504837    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrstj\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-kube-api-access-xrstj\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505112 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504900    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505112 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504946    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505112 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.504969    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505287 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.505180    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505287 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.505238    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505471 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.505451    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505532 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.505467    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ff8aad-679b-4686-9b94-fb4441696b67-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.505610 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.505596    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.507074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.507048    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.507315 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.507298    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.521026 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.521004    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:45.563582 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.562543    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.563582 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.562703    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrstj\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-kube-api-access-xrstj\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.724071 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.724029    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:56:45.843891 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.843805    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:45.907211 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.906999    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"]
Oct 30 21:56:45.911243 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:56:45.911090    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84239b3b_2157_4921_aacd_ed21bc708701.slice/crio-3d706d1f5ef1eac70f37d87bef26720fdd0659282d731e585dc0e6e554daf57d WatchSource:0}: Error finding container 3d706d1f5ef1eac70f37d87bef26720fdd0659282d731e585dc0e6e554daf57d: Status 404 returned error can't find the container with id 3d706d1f5ef1eac70f37d87bef26720fdd0659282d731e585dc0e6e554daf57d
Oct 30 21:56:45.913691 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.913648    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:45.913774 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.913743    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:45.913818 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:45.913788    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:46.077585 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.074957    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"]
Oct 30 21:56:46.078108 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.078074    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" containerID="cri-o://f025585e80d75292ea27f51a8fec4512c854a5ca76316a7d6e11e7e92b5672f2" gracePeriod=30
Oct 30 21:56:46.179509 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.179467    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" event={"ID":"84239b3b-2157-4921-aacd-ed21bc708701","Type":"ContainerStarted","Data":"38ca2bca61288b5d49ab8b49136fa58d4e66a5cae1becb63a05fb9e57ff33028"}
Oct 30 21:56:46.179722 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.179698    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" event={"ID":"84239b3b-2157-4921-aacd-ed21bc708701","Type":"ContainerStarted","Data":"3d706d1f5ef1eac70f37d87bef26720fdd0659282d731e585dc0e6e554daf57d"}
Oct 30 21:56:46.239304 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.237565    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"]
Oct 30 21:56:46.242135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.242089    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:46.242272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.242185    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:46.242272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.242234    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:46.330135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.328176    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podStartSLOduration=2.328159726 podStartE2EDuration="2.328159726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:46.327313207 +0000 UTC m=+1354.605006719" watchObservedRunningTime="2026-10-30 21:56:46.328159726 +0000 UTC m=+1354.605853200"
Oct 30 21:56:46.530370 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:46.527473    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:47.281717 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.281672    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" event={"ID":"a0ff8aad-679b-4686-9b94-fb4441696b67","Type":"ContainerStarted","Data":"c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe"}
Oct 30 21:56:47.282314 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.282283    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" event={"ID":"a0ff8aad-679b-4686-9b94-fb4441696b67","Type":"ContainerStarted","Data":"e8d9e0aa76b90dd2f43ba9d7f631b5853c5b5f0f46aa4399107d48024a310d49"}
Oct 30 21:56:47.525375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.516181    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podStartSLOduration=2.51616014 podStartE2EDuration="2.51616014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:47.462862118 +0000 UTC m=+1355.740555589" watchObservedRunningTime="2026-10-30 21:56:47.51616014 +0000 UTC m=+1355.793853614"
Oct 30 21:56:47.529376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.527483    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:47.529376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.527541    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:47.865378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.864274    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:47.865378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.864342    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:47.941375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:47.931522    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:48.230380 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.229477    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.435376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.407465    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.528446 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.523487    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:48.528792 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.528752    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:48.622569 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.622456    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.718635 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.718591    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": context deadline exceeded"
Oct 30 21:56:48.831665 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.831500    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.847529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.839752    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.847529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.844653    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:48.847529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:48.844698    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:49.238374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.225792    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:56:49.238374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.226137    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.238374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.226748    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.308397 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.276492    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:56:49.308397 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.277619    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:56:49.335389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.313436    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.335389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.315820    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:56:49.408376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.407459    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.483196 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.428777    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.484009 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.438976    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.484429 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.484376    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:56:49.632375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.631208    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:49.632375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.631291    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:49.821402 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.820485    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.834726 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.833434    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.856342 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.854374    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:56:49.860534 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.860473    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:49.860798 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.860732    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:49.957056 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:49.892904    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.095374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.070082    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:56:50.534777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.302006    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.534777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.362967    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.534777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.363000    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.534777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.508739    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:56:50.534777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.508801    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:56:50.534777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.363027    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.534777 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.408002    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.549376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.537779    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:50.553469 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.553425    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:50.557434 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.491991    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.408049    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.408070    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.561379    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.561452    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.561499    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.561550    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.491956    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.577373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.563276    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:56:50.839631 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.839524    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.883384 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:50.858232    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:51.036429 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.032766    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:51.036429 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.032467    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": context deadline exceeded"
Oct 30 21:56:51.036429 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.007463    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.375894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.375844    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.712791 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.712744    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.714935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.714883    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.728334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.718408    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.728334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.690417    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.728334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.720852    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.728334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.721017    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.728334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.722039    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.728334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.722259    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.728334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.722281    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.876494 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.853853    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:51.876494 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.853918    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:51.876494 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.862425    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.901095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.901051    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:51.901392 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.901339    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.908510 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:51.908476    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:52.024589 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.024484    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:52.044629 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.044578    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.104414 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.104101    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.152377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.124193    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.516377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.491234    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:52.517531 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.517484    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.743435 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.713763    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.882073 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.719870    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.882819 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.723043    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:56:52.947068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.947018    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Oct 30 21:56:52.947269 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.748534    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:56:52.962289 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.748871    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:52.998569 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.723218    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.999454 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.863523    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.000463 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.000435    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.000707 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.000648    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:53.000991 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:52.749073    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.024377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.024315    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:56:53.132553 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.132504    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:53.132886 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.132852    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.146516 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.139598    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:56:53.196733 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.196680    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.242699 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.242653    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:56:53.276459 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.271443    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:53.378385 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.372413    5561 generic.go:356] "Generic (PLEG): container finished" podID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerID="f025585e80d75292ea27f51a8fec4512c854a5ca76316a7d6e11e7e92b5672f2" exitCode=0
Oct 30 21:56:53.378713 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.378687    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" event={"ID":"df6de057-3fef-4a72-b7f3-e685586bd27d","Type":"ContainerDied","Data":"f025585e80d75292ea27f51a8fec4512c854a5ca76316a7d6e11e7e92b5672f2"}
Oct 30 21:56:53.568940 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.568856    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.609383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.608336    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:53.629641 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:56:53.629595    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:56:53.895471 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.891306    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.895471 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.891380    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:53.970407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.938195    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.970407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.938634    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.970407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.938922    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:56:53.980390 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.979248    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.980390 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:53.979763    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.006698 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.006641    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.013154 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.013110    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.017149 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.017109    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:56:54.040579 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.040539    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:56:54.326039 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.326000    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:56:54.342400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.342198    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:56:54.343209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.342655    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:56:54.343209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.342699    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:56:54.343209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.342730    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:56:54.343209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.342761    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:56:54.343209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.342789    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:56:54.343209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.342818    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:56:54.371388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.367950    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:56:54.379840 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.379812    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:56:54.401730 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.401689    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-token\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-token\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.401903 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.401807    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-envoy\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.401903 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.401846    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-socket\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.402019 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.401974    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-data\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.402085 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.402069    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-kube-api-access-fzfzc\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-kube-api-access-fzfzc\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.402157 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.402135    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-credential-socket\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.407221 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.407158    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-podinfo\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.407368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.407330    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-certs\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.407446 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.407390    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df6de057-3fef-4a72-b7f3-e685586bd27d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6de057-3fef-4a72-b7f3-e685586bd27d-istiod-ca-cert\") pod \"df6de057-3fef-4a72-b7f3-e685586bd27d\" (UID: \"df6de057-3fef-4a72-b7f3-e685586bd27d\") "
Oct 30 21:56:54.409032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.402682    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-data" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:54.409032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.407034    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-credential-socket" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:54.409032 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.407089    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-socket" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:54.409534 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.409491    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:56:54.411895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.411873    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-certs" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:54.414543 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.414518    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df6de057-3fef-4a72-b7f3-e685586bd27d-istiod-ca-cert" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:54.415820 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.415794    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-envoy" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:54.415995 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.415971    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-token" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:54.416216 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.416199    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-kube-api-access-fzfzc" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "kube-api-access-fzfzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:54.418778 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.418758    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-podinfo" pod "df6de057-3fef-4a72-b7f3-e685586bd27d" (UID: "df6de057-3fef-4a72-b7f3-e685586bd27d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:56:54.433170 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.433151    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:56:54.435656 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.435636    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"
Oct 30 21:56:54.435755 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.435636    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" event={"ID":"df6de057-3fef-4a72-b7f3-e685586bd27d","Type":"ContainerDied","Data":"e27e9f87893b24168838b397b67903c1f17ba6c6387228ca49b0809956ab2d87"}
Oct 30 21:56:54.435816 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.435770    5561 scope.go:122] "RemoveContainer" containerID="f025585e80d75292ea27f51a8fec4512c854a5ca76316a7d6e11e7e92b5672f2"
Oct 30 21:56:54.442699 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.440807    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:56:54.510962 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.510932    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511106 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511095    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzfzc\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-kube-api-access-fzfzc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511185 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511176    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511234    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511300 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511291    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511345    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6de057-3fef-4a72-b7f3-e685586bd27d-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511429    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511491 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511483    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.511538 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.511531    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6de057-3fef-4a72-b7f3-e685586bd27d-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:54.521433 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.521407    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:54.521530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.521473    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:54.811444 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.809210    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:54.811444 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.809264    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.902395 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.897577    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:54.902395 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:54.897637    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:55.273374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:55.269427    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" podUID="9a9ea5e58d330910c40e8b399ca003fd" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:55.522677 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:55.522235    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:55.522934 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:55.522903    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:55.857486 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:55.857449    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:55.920269 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:55.920202    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:55.928808 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:55.928768    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"]
Oct 30 21:56:55.929064 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:55.929045    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt"]
Oct 30 21:56:56.521407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:56.521299    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:56.521557 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:56.521502    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:56.916410 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:56.915554    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:56.916714 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:56.916678    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:57.521506 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:57.521473    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:56:57.521982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:57.521524    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:56:57.844825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:57.844725    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:56:57.844825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:57.844791    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:56:57.850873 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:57.850843    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" path="/var/lib/kubelet/pods/df6de057-3fef-4a72-b7f3-e685586bd27d/volumes"
Oct 30 21:56:59.095979 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:59.095944    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:59.129431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:59.121492    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:59.136950 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:59.136912    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 21:56:59.299267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:59.299237    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:59.300271 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:59.300244    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:56:59.301295 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:56:59.301262    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 21:57:19.053104 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.053066    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"]
Oct 30 21:57:19.113371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.096774    5561 cpu_manager.go:414] "containerMap: removing container" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy"
Oct 30 21:57:19.204701 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.204671    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy"
Oct 30 21:57:19.205217 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.205187    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="df6de057-3fef-4a72-b7f3-e685586bd27d" containerName="istio-proxy"
Oct 30 21:57:19.220170 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.220135    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"]
Oct 30 21:57:19.220334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.220264    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.322780 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.277780    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f05b6443-openshift-default-dockercfg-7zdmz\""
Oct 30 21:57:19.443230 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443196    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-data\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443441 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443261    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-podinfo\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443441 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443405    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istiod-ca-cert\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443568 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443468    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-credential-socket\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443568 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443541    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-token\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443636 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443622    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-envoy\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443729 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443715    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-socket\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443834 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443820    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhr7s\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-kube-api-access-fhr7s\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.443894 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.443878    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-certs\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.544825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.544792    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhr7s\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-kube-api-access-fhr7s\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.544825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.544837    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-certs\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545087 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.544900    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-data\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545087 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.544978    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-podinfo\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545087 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545038    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istiod-ca-cert\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545269 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545243    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-credential-socket\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545335 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545283    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-token\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545448 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545338    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-envoy\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545508 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545362    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-data\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545508 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545462    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-socket\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545657 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545632    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-credential-socket\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545721 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545703    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-certs\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545780 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545750    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-socket\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.545837 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.545800    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istiod-ca-cert\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.547458 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.547438    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-envoy\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.547695 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.547674    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-podinfo\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.632373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.627671    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhr7s\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-kube-api-access-fhr7s\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.650724 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.644728    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-token\") pod \"e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") " pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:19.651084 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:19.651051    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:20.090197 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:20.090164    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"]
Oct 30 21:57:20.091884 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:57:20.091856    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdae184_f33e_4f49_9d68_85cc9aecb2b9.slice/crio-5152fbedb4b66d37e20546c2a3bf0deabc9562badfa8d2378a74a4efb70fb813 WatchSource:0}: Error finding container 5152fbedb4b66d37e20546c2a3bf0deabc9562badfa8d2378a74a4efb70fb813: Status 404 returned error can't find the container with id 5152fbedb4b66d37e20546c2a3bf0deabc9562badfa8d2378a74a4efb70fb813
Oct 30 21:57:20.094830 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:20.094791    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:20.094905 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:20.094878    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:20.094958 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:20.094917    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:20.752665 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:20.752618    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" event={"ID":"4cdae184-f33e-4f49-9d68-85cc9aecb2b9","Type":"ContainerStarted","Data":"23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53"}
Oct 30 21:57:20.752824 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:20.752722    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" event={"ID":"4cdae184-f33e-4f49-9d68-85cc9aecb2b9","Type":"ContainerStarted","Data":"5152fbedb4b66d37e20546c2a3bf0deabc9562badfa8d2378a74a4efb70fb813"}
Oct 30 21:57:21.663851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:21.663815    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:21.672972 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:21.672943    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:21.838420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:21.802428    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:21.838420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:21.814657    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:57:21.930491 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:21.929564    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podStartSLOduration=3.929542306 podStartE2EDuration="3.929542306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:20.978856534 +0000 UTC m=+1389.256550006" watchObservedRunningTime="2026-10-30 21:57:21.929542306 +0000 UTC m=+1390.207235778"
Oct 30 21:57:22.723376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:22.716945    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:22.920237 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:22.920188    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.051427 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.046448    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.411269 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:57:23.382583    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:57:23.437936 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.437887    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.458528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.458455    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.513510 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.513463    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.718810 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.718714    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.719446 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.719044    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.721657 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.720081    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:57:23.731804 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.731777    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:57:23.738368 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.732275    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.777939 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:23.777903    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.040637 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.032540    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:57:24.308566 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.308448    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:24.308884 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.308850    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.334396 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.312434    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.450515 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.450466    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.452230 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.452202    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:57:24.487400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.461417    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:57:24.487400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.468526    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:57:24.537743 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.537702    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:57:24.562035 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.561939    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:57:24.729374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.725539    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.817168 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.733430    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.829418 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.736476    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.838439 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.814627    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.839656 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.816780    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:57:24.860388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.818150    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:57:24.860388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.849610    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:57:24.860388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.849662    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:57:24.860388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:24.849702    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:57:25.079427 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.063950    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.079427 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.065235    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:57:25.112484 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.106592    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.154838 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.115932    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.328376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.322461    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.328376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.323972    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:57:25.348431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.347538    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:25.348431 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.347620    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.379676 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.379635    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": context deadline exceeded"
Oct 30 21:57:25.492488 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.491483    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.493797 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.492761    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.580374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.569049    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.664692 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.664657    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:25.664981 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.664953    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.818262 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.818103    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.819654 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.819554    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.844859 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.842759    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.844859 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.843641    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.844859 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.844303    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.844859 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:25.844581    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.076923 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.076828    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:57:26.163388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.163328    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.164952 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.163857    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:57:26.169758 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.169735    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 21:57:26.330756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.330663    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.492018 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.491977    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.582249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.581442    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 21:57:26.846499 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.845156    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.846499 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.846326    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:57:26.848856 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.847339    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.848856 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.847910    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 21:57:26.853609 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.853580    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.874377 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.872402    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:57:26.905341 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.905308    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:57:26.911184 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.911161    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:57:26.929280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.929164    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:26.929280 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.929219    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.959209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:26.958447    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:57:27.082761 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.082630    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:27.272375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.262301    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:27.272375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.264084    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 21:57:27.272375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.267614    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:57:27.286285 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.286256    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:57:27.318666 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.318638    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 21:57:27.319131 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.319111    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:57:27.337837 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:27.337791    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 21:57:28.376110 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.376075    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"]
Oct 30 21:57:28.382151 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.382118    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.444332 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.444303    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-be51942b-openshift-default-dockercfg-lmbvq\""
Oct 30 21:57:28.444843 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.444812    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"]
Oct 30 21:57:28.445984 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.445657    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-podinfo\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.446094 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.446019    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zckhw\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-kube-api-access-zckhw\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.446094 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.446063    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-envoy\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.446201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.446100    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-socket\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.446201 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.446140    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-data\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.446438 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.446414    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-token\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.447383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.447338    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-credential-socket\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.447541 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.447403    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-certs\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.447541 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.447437    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istiod-ca-cert\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.565733    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-credential-socket\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.565795    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-certs\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.565835    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istiod-ca-cert\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.566039    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-podinfo\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.566075    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zckhw\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-kube-api-access-zckhw\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.566127    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-envoy\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.566170    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-socket\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.566220    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-data\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.566313    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-token\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.567103    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-credential-socket\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.569136    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istiod-ca-cert\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.586371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.582207    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-podinfo\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.622397 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.603735    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-data\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.622397 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.604389    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-socket\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.650872 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.650785    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"]
Oct 30 21:57:28.703373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.675657    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-envoy\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.703373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.685971    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-token\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.703373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.694475    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-certs\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.715379 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.711640    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:28.769373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.765103    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"]
Oct 30 21:57:28.784373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.781194    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zckhw\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-kube-api-access-zckhw\") pod \"e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") " pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:28.784373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.782437    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd030ec-a67e-4958-908a-c475d14aea57-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:28.784373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.782813    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcrhk\" (UniqueName: \"kubernetes.io/projected/1cd030ec-a67e-4958-908a-c475d14aea57-kube-api-access-xcrhk\") pod \"payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:28.791369 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.787751    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f05b6443-dockercfg-phmkv\""
Oct 30 21:57:28.801462 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.801434    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f05b6443\""
Oct 30 21:57:28.893148 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.893110    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcrhk\" (UniqueName: \"kubernetes.io/projected/1cd030ec-a67e-4958-908a-c475d14aea57-kube-api-access-xcrhk\") pod \"payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:28.893524 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.893500    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd030ec-a67e-4958-908a-c475d14aea57-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:28.896156 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.896097    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd030ec-a67e-4958-908a-c475d14aea57-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:28.974852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:28.965834    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcrhk\" (UniqueName: \"kubernetes.io/projected/1cd030ec-a67e-4958-908a-c475d14aea57-kube-api-access-xcrhk\") pod \"payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:29.076388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:29.070682    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:29.211771 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:29.211736    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:29.548287 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:29.548255    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"]
Oct 30 21:57:29.548921 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:57:29.548896    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be698b7_5a83_4d30_8ace_917f0ac6fe1a.slice/crio-6030e4e49b07185bdf2727fcec8318dfa92d010df68caa7ab00e588766b2e17b WatchSource:0}: Error finding container 6030e4e49b07185bdf2727fcec8318dfa92d010df68caa7ab00e588766b2e17b: Status 404 returned error can't find the container with id 6030e4e49b07185bdf2727fcec8318dfa92d010df68caa7ab00e588766b2e17b
Oct 30 21:57:29.551596 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:29.551564    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:29.551672 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:29.551629    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:29.551672 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:29.551658    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:29.662340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:29.662309    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"]
Oct 30 21:57:29.662683 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:57:29.662652    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd030ec_a67e_4958_908a_c475d14aea57.slice/crio-3ba8740e0ace040276abe2086ee928b0dace21922fb7d63fccddc0fe4a905c05 WatchSource:0}: Error finding container 3ba8740e0ace040276abe2086ee928b0dace21922fb7d63fccddc0fe4a905c05: Status 404 returned error can't find the container with id 3ba8740e0ace040276abe2086ee928b0dace21922fb7d63fccddc0fe4a905c05
Oct 30 21:57:30.391041 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:30.391005    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x" event={"ID":"1cd030ec-a67e-4958-908a-c475d14aea57","Type":"ContainerStarted","Data":"3da99acf1314a118cbf8aa676a737f15f81eed369dcfa00255c254b3cc6a8852"}
Oct 30 21:57:30.391229 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:30.391067    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x" event={"ID":"1cd030ec-a67e-4958-908a-c475d14aea57","Type":"ContainerStarted","Data":"3ba8740e0ace040276abe2086ee928b0dace21922fb7d63fccddc0fe4a905c05"}
Oct 30 21:57:30.391461 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:30.391434    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:30.393186 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:30.393159    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" event={"ID":"0be698b7-5a83-4d30-8ace-917f0ac6fe1a","Type":"ContainerStarted","Data":"383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8"}
Oct 30 21:57:30.393310 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:30.393201    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" event={"ID":"0be698b7-5a83-4d30-8ace-917f0ac6fe1a","Type":"ContainerStarted","Data":"6030e4e49b07185bdf2727fcec8318dfa92d010df68caa7ab00e588766b2e17b"}
Oct 30 21:57:30.508960 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:30.508905    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x" podStartSLOduration=2.50889186 podStartE2EDuration="2.50889186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:30.506901212 +0000 UTC m=+1398.784594688" watchObservedRunningTime="2026-10-30 21:57:30.50889186 +0000 UTC m=+1398.786585330"
Oct 30 21:57:30.626564 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:30.626516    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podStartSLOduration=2.626503309 podStartE2EDuration="2.626503309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:30.625896396 +0000 UTC m=+1398.903589867" watchObservedRunningTime="2026-10-30 21:57:30.626503309 +0000 UTC m=+1398.904196780"
Oct 30 21:57:31.072883 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:31.072851    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:31.077639 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:31.077608    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:31.400206 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:31.400171    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:31.400984 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:31.400961    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:57:37.278652 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.278620    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"]
Oct 30 21:57:37.282937 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.282915    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:37.340077 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.340050    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"]
Oct 30 21:57:37.345292 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.345263    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77946a-7423-44c7-855f-45c34ee9ddb8-plugins-config-volume\") pod \"payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") " pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:37.345669 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.345639    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5fjj\" (UniqueName: \"kubernetes.io/projected/9c77946a-7423-44c7-855f-45c34ee9ddb8-kube-api-access-g5fjj\") pod \"payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") " pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:37.386398 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.386337    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-be51942b-dockercfg-224x9\""
Oct 30 21:57:37.386528 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.386415    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-be51942b\""
Oct 30 21:57:37.447213 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.447182    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5fjj\" (UniqueName: \"kubernetes.io/projected/9c77946a-7423-44c7-855f-45c34ee9ddb8-kube-api-access-g5fjj\") pod \"payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") " pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:37.447334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.447277    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77946a-7423-44c7-855f-45c34ee9ddb8-plugins-config-volume\") pod \"payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") " pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:37.448059 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.448034    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77946a-7423-44c7-855f-45c34ee9ddb8-plugins-config-volume\") pod \"payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") " pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:37.653432 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.603251    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5fjj\" (UniqueName: \"kubernetes.io/projected/9c77946a-7423-44c7-855f-45c34ee9ddb8-kube-api-access-g5fjj\") pod \"payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") " pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:37.654485 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:37.654101    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:38.302030 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:38.302006    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"]
Oct 30 21:57:38.302493 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:57:38.302469    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c77946a_7423_44c7_855f_45c34ee9ddb8.slice/crio-191a6558f0b606259ccb1ee83596e0d1a16cefe593faf2b9d35fbeb7cce9ce51 WatchSource:0}: Error finding container 191a6558f0b606259ccb1ee83596e0d1a16cefe593faf2b9d35fbeb7cce9ce51: Status 404 returned error can't find the container with id 191a6558f0b606259ccb1ee83596e0d1a16cefe593faf2b9d35fbeb7cce9ce51
Oct 30 21:57:38.752195 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:38.752161    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh" event={"ID":"9c77946a-7423-44c7-855f-45c34ee9ddb8","Type":"ContainerStarted","Data":"1ee2b9f28e6d153d041c64e3a99d9ecc2b3b498b3545a3c962bac3d6d4df2050"}
Oct 30 21:57:38.752383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:38.752222    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh" event={"ID":"9c77946a-7423-44c7-855f-45c34ee9ddb8","Type":"ContainerStarted","Data":"191a6558f0b606259ccb1ee83596e0d1a16cefe593faf2b9d35fbeb7cce9ce51"}
Oct 30 21:57:38.752383 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:38.752268    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:38.873911 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:38.873842    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh" podStartSLOduration=1.8738260169999998 podStartE2EDuration="1.873826017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:38.871528778 +0000 UTC m=+1407.149222248" watchObservedRunningTime="2026-10-30 21:57:38.873826017 +0000 UTC m=+1407.151519489"
Oct 30 21:57:41.403206 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:41.403178    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:57:45.129708 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.129670    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"]
Oct 30 21:57:45.139750 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.136215    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.236122 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.236090    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e585c53f-dockercfg-nrwkz\""
Oct 30 21:57:45.236489 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.236468    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e585c53f\""
Oct 30 21:57:45.258824 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.253447    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"]
Oct 30 21:57:45.383144 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.383054    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7400ac-d88c-4cf2-ace8-2ac12d762714-plugins-config-volume\") pod \"payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") " pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.399869 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.399830    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5kkd\" (UniqueName: \"kubernetes.io/projected/3f7400ac-d88c-4cf2-ace8-2ac12d762714-kube-api-access-z5kkd\") pod \"payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") " pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.479544 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.478422    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"]
Oct 30 21:57:45.484551 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.484077    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"]
Oct 30 21:57:45.484551 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.484125    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"]
Oct 30 21:57:45.484551 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.484184    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.489074 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.489049    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.501646 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.501617    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7400ac-d88c-4cf2-ace8-2ac12d762714-plugins-config-volume\") pod \"payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") " pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.501850 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.501829    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5kkd\" (UniqueName: \"kubernetes.io/projected/3f7400ac-d88c-4cf2-ace8-2ac12d762714-kube-api-access-z5kkd\") pod \"payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") " pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.508003 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.507977    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7400ac-d88c-4cf2-ace8-2ac12d762714-plugins-config-volume\") pod \"payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") " pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.569740 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.569707    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"]
Oct 30 21:57:45.577779 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.577755    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5kkd\" (UniqueName: \"kubernetes.io/projected/3f7400ac-d88c-4cf2-ace8-2ac12d762714-kube-api-access-z5kkd\") pod \"payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") " pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.603172 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603140    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603298 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603213    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603307    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603400 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603335    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba9bb7d-753c-4b57-93b6-0984dac30f06-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603524 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603455    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603524 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603494    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5dw\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-kube-api-access-8x5dw\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603643 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603530    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfl4p\" (UniqueName: \"kubernetes.io/projected/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-kube-api-access-pfl4p\") pod \"payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.603643 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603600    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603643 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603620    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.603802 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603659    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.603802 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.603772    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.704797 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.704724    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.704942 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.704820    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.704942 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.704866    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705018 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.704986    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba9bb7d-753c-4b57-93b6-0984dac30f06-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705095    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705181 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705153    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5dw\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-kube-api-access-8x5dw\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705240 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705203    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfl4p\" (UniqueName: \"kubernetes.io/projected/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-kube-api-access-pfl4p\") pod \"payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.705295 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705247    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705295 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705273    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705295 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705275    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705474 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705318    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705474 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705341    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.705474 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705449    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705622 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705580    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705685 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705662    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.705844 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.705821    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba9bb7d-753c-4b57-93b6-0984dac30f06-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.706140 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.706115    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.707702 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.707676    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.707801 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.707692    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.767336 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.767308    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.767735 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.767718    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfl4p\" (UniqueName: \"kubernetes.io/projected/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-kube-api-access-pfl4p\") pod \"payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.769980 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.769960    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5dw\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-kube-api-access-8x5dw\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:45.772834 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.772814    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:45.800331 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.800309    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:45.808660 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:45.808638    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:46.167582 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.167556    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"]
Oct 30 21:57:46.168242 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:57:46.168215    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7400ac_d88c_4cf2_ace8_2ac12d762714.slice/crio-379b0c5d367761a87054d60f717bcbf9c8fc1a68da4be286d185826a5eb9de92 WatchSource:0}: Error finding container 379b0c5d367761a87054d60f717bcbf9c8fc1a68da4be286d185826a5eb9de92: Status 404 returned error can't find the container with id 379b0c5d367761a87054d60f717bcbf9c8fc1a68da4be286d185826a5eb9de92
Oct 30 21:57:46.218173 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:57:46.218147    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf485eae_fedb_41c6_bb72_7c6ea2d4cc19.slice/crio-7e4cb365224689ba21ed33ceaf9cbdb664fa60a40432fe4fac03a67cd9c79eef WatchSource:0}: Error finding container 7e4cb365224689ba21ed33ceaf9cbdb664fa60a40432fe4fac03a67cd9c79eef: Status 404 returned error can't find the container with id 7e4cb365224689ba21ed33ceaf9cbdb664fa60a40432fe4fac03a67cd9c79eef
Oct 30 21:57:46.218511 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.218488    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"]
Oct 30 21:57:46.277505 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:57:46.277474    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba9bb7d_753c_4b57_93b6_0984dac30f06.slice/crio-61ac9016334dcb48ce2a3e32b79e65fb31d6d77d8eae0278c1bae7f69f77f01b WatchSource:0}: Error finding container 61ac9016334dcb48ce2a3e32b79e65fb31d6d77d8eae0278c1bae7f69f77f01b: Status 404 returned error can't find the container with id 61ac9016334dcb48ce2a3e32b79e65fb31d6d77d8eae0278c1bae7f69f77f01b
Oct 30 21:57:46.279364 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.279324    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"]
Oct 30 21:57:46.280851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.280806    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:46.280925 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.280912    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:46.280971 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.280959    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:57:46.440778 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.440700    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:57:46.874938 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.874901    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" event={"ID":"bba9bb7d-753c-4b57-93b6-0984dac30f06","Type":"ContainerStarted","Data":"8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921"}
Oct 30 21:57:46.875197 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.875171    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" event={"ID":"bba9bb7d-753c-4b57-93b6-0984dac30f06","Type":"ContainerStarted","Data":"61ac9016334dcb48ce2a3e32b79e65fb31d6d77d8eae0278c1bae7f69f77f01b"}
Oct 30 21:57:46.877799 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.877772    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm" event={"ID":"af485eae-fedb-41c6-bb72-7c6ea2d4cc19","Type":"ContainerStarted","Data":"4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb"}
Oct 30 21:57:46.877901 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.877812    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm" event={"ID":"af485eae-fedb-41c6-bb72-7c6ea2d4cc19","Type":"ContainerStarted","Data":"7e4cb365224689ba21ed33ceaf9cbdb664fa60a40432fe4fac03a67cd9c79eef"}
Oct 30 21:57:46.878132 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.878112    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:57:46.885899 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.885875    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v" event={"ID":"3f7400ac-d88c-4cf2-ace8-2ac12d762714","Type":"ContainerStarted","Data":"ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f"}
Oct 30 21:57:46.886052 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.886033    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v" event={"ID":"3f7400ac-d88c-4cf2-ace8-2ac12d762714","Type":"ContainerStarted","Data":"379b0c5d367761a87054d60f717bcbf9c8fc1a68da4be286d185826a5eb9de92"}
Oct 30 21:57:46.886169 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:46.886157    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:47.207447 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:47.206856    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v" podStartSLOduration=2.206836303 podStartE2EDuration="2.206836303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:47.192148718 +0000 UTC m=+1415.469842219" watchObservedRunningTime="2026-10-30 21:57:47.206836303 +0000 UTC m=+1415.484529775"
Oct 30 21:57:47.212371 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:47.208616    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podStartSLOduration=2.208591464 podStartE2EDuration="2.208591464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:47.030651128 +0000 UTC m=+1415.308344613" watchObservedRunningTime="2026-10-30 21:57:47.208591464 +0000 UTC m=+1415.486284935"
Oct 30 21:57:47.345391 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:47.343065    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm" podStartSLOduration=2.343043786 podStartE2EDuration="2.343043786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:47.341909519 +0000 UTC m=+1415.619602991" watchObservedRunningTime="2026-10-30 21:57:47.343043786 +0000 UTC m=+1415.620737259"
Oct 30 21:57:47.820375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:47.811987    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:47.820375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:47.813795    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:47.820375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:47.813933    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:48.701729 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:48.701632    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:48.812869 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:48.812831    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:48.813926 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:48.813891    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:48.861378 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:48.860422    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.328975 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.328455    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.349407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.342834    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.828062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.827475    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.851244 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.841661    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.856375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.851512    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.888234 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.856593    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.898372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.888557    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.898372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.888811    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:57:49.958627 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:49.920444    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:50.092531 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.051966    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:50.092531 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.052026    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:50.127748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.110228    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:57:50.157365 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.154547    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:57:50.277021 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.267695    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:50.330278 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.328472    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:50.330278 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.328988    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:57:50.330278 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.329248    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:57:50.393785 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.376029    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:50.393785 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.379457    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:57:50.609078 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.582652    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:57:50.844432 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.843911    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:57:50.865373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.845332    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:57:50.865373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.863738    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:50.865373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.863803    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:50.865373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.863873    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:50.865373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.864154    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:57:50.883489 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.881723    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:51.082661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.892468    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:51.082661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.892827    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:57:51.082661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.893141    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:57:51.082661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:50.959040    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:51.082661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.001493    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 21:57:51.082661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.001561    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:57:51.108962 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.085092    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 21:57:51.108962 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.104181    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 21:57:51.168792 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.168762    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 21:57:51.208871 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.208843    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 21:57:51.210095 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.210073    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 21:57:51.213136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.213100    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 21:57:51.809608 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.809577    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:51.809781 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:51.809623    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:52.809285 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:52.809253    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:52.809716 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:52.809309    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:52.998213 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:57:52.998170    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:57:53.809308 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:53.809272    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:53.809728 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:53.809334    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:54.809236 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:54.809193    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:57:54.809497 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:54.809243    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 21:57:55.857976 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:55.857947    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:55.858812 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:55.858785    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:55.860572 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:55.860545    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 21:57:57.956712 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:57.956684    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 21:57:57.967339 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:57:57.967317    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 21:58:10.236522 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.236489    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"]
Oct 30 21:58:10.240974 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.240953    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.294716 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.294687    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f05b6443-serving-cert\""
Oct 30 21:58:10.331878 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.331843    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.332153 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.332124    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-metrics-certs\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.332304 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.332278    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwpzv\" (UniqueName: \"kubernetes.io/projected/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-kube-api-access-vwpzv\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.332462 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.332318    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-openshift-service-ca\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.348860 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.348837    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"]
Oct 30 21:58:10.433825 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.433779    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwpzv\" (UniqueName: \"kubernetes.io/projected/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-kube-api-access-vwpzv\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.433975 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.433832    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-openshift-service-ca\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.433975 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.433926    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.434097 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:10.434080    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-serving-cert: secret "maas-api-e2e-ait-f05b6443-serving-cert" not found
Oct 30 21:58:10.434133 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.434099    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-metrics-certs\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.434173 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:10.434152    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls podName:cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8 nodeName:}" failed. No retries permitted until 2026-10-30 21:58:10.934131487 +0000 UTC m=+1439.211824942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls") pod "maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf" (UID: "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8") : secret "maas-api-e2e-ait-f05b6443-serving-cert" not found
Oct 30 21:58:10.434490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.434469    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-openshift-service-ca\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.436750 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.436723    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-metrics-certs\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.545477 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.542816    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwpzv\" (UniqueName: \"kubernetes.io/projected/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-kube-api-access-vwpzv\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.960605 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.960566    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:10.962917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:10.962894    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls\") pod \"maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:11.155282 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:11.155254    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:11.546057 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:58:11.546024    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf424c1_3d9c_4fb6_b946_3cc46dd27ec8.slice/crio-df24222f247f174f3c2b47e6f515496253191827a4cd2b8d067484faf3e257b5 WatchSource:0}: Error finding container df24222f247f174f3c2b47e6f515496253191827a4cd2b8d067484faf3e257b5: Status 404 returned error can't find the container with id df24222f247f174f3c2b47e6f515496253191827a4cd2b8d067484faf3e257b5
Oct 30 21:58:11.547048 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:11.547020    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"]
Oct 30 21:58:11.547941 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:11.547918    5561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:58:11.766410 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:11.766339    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf" event={"ID":"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8","Type":"ContainerStarted","Data":"df24222f247f174f3c2b47e6f515496253191827a4cd2b8d067484faf3e257b5"}
Oct 30 21:58:12.774416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:12.774375    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf" event={"ID":"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8","Type":"ContainerStarted","Data":"b0dbb1ccce264a35156489a57b61daf1350512a95a8079849d34312d18e22e55"}
Oct 30 21:58:12.774877 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:12.774497    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:12.921965 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:12.921917    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf" podStartSLOduration=1.863212941 podStartE2EDuration="2.921903178s" totalImagesPullingTime="1.058690237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:12.919782918 +0000 UTC m=+1441.197476391" watchObservedRunningTime="2026-10-30 21:58:12.921903178 +0000 UTC m=+1441.199596649"
Oct 30 21:58:16.926828 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:16.926794    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"]
Oct 30 21:58:16.931340 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:16.931315    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:16.998705 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:16.998682    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xn2np\""
Oct 30 21:58:17.007446 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007423    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007511 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007465    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2np5\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-kube-api-access-p2np5\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007562 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007538    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007562    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec4d573a-ec69-4842-b77f-8c3932e1efef-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007597 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007588    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007663 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007619    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007760 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007735    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007813 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007800    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.007851 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.007824    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.066915 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.066883    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"]
Oct 30 21:58:17.109830 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.109795    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2np5\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-kube-api-access-p2np5\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.109996 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.109870    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.109996 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.109901    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec4d573a-ec69-4842-b77f-8c3932e1efef-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.109996 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.109937    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.109996 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.109993    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.110291 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.110077    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.110291 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.110138    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.110291 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.110167    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.110473 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.110307    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.110473 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.110458    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.110699 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.110676    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.111316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.110972    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.111316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.111083    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec4d573a-ec69-4842-b77f-8c3932e1efef-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.111316 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.111186    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.115427 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.113509    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.115427 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.113705    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.184370 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.184298    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.184476 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.184459    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2np5\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-kube-api-access-p2np5\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bmmjn\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.246261 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.246234    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:17.688154 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.688122    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"]
Oct 30 21:58:17.688515 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:58:17.688488    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4d573a_ec69_4842_b77f_8c3932e1efef.slice/crio-0b65835b9489d78c26086c7efd73d53932f9d331b623f21aa48b2ebfc2578b29 WatchSource:0}: Error finding container 0b65835b9489d78c26086c7efd73d53932f9d331b623f21aa48b2ebfc2578b29: Status 404 returned error can't find the container with id 0b65835b9489d78c26086c7efd73d53932f9d331b623f21aa48b2ebfc2578b29
Oct 30 21:58:17.690942 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.690901    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:58:17.691033 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.690996    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:58:17.691033 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.691026    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:58:17.810125 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.810083    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" event={"ID":"ec4d573a-ec69-4842-b77f-8c3932e1efef","Type":"ContainerStarted","Data":"dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923"}
Oct 30 21:58:17.810287 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.810153    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" event={"ID":"ec4d573a-ec69-4842-b77f-8c3932e1efef","Type":"ContainerStarted","Data":"0b65835b9489d78c26086c7efd73d53932f9d331b623f21aa48b2ebfc2578b29"}
Oct 30 21:58:17.936890 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:17.936787    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podStartSLOduration=1.9367702439999999 podStartE2EDuration="1.936770244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:17.934757639 +0000 UTC m=+1446.212451135" watchObservedRunningTime="2026-10-30 21:58:17.936770244 +0000 UTC m=+1446.214463716"
Oct 30 21:58:18.250469 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:18.247682    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:18.254388 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:18.254333    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:18.254694 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:18.254661    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:18.787605 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:18.787575    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 21:58:19.256374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:19.254441    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:19.256374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:19.254500    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:20.266376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.258637    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:20.266376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.258705    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:20.771785 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.771746    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"]
Oct 30 21:58:20.778413 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.778387    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:20.831661 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.831631    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"]
Oct 30 21:58:20.906684 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.906647    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1c052f-90b4-47c6-a636-dd77c751cba9-openshift-service-ca\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:20.906865 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.906823    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9rjt\" (UniqueName: \"kubernetes.io/projected/9e1c052f-90b4-47c6-a636-dd77c751cba9-kube-api-access-v9rjt\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:20.906937 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.906882    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:20.907041 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.907024    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-metrics-certs\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:20.909636 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:20.909608    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-be51942b-serving-cert\""
Oct 30 21:58:21.054425 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.048979    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1c052f-90b4-47c6-a636-dd77c751cba9-openshift-service-ca\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.054425 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.049213    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9rjt\" (UniqueName: \"kubernetes.io/projected/9e1c052f-90b4-47c6-a636-dd77c751cba9-kube-api-access-v9rjt\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.054425 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.049274    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.054425 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.049384    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-metrics-certs\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.058382 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.056825    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1c052f-90b4-47c6-a636-dd77c751cba9-openshift-service-ca\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.058382 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:21.056958    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-serving-cert: secret "maas-api-e2e-ait-be51942b-serving-cert" not found
Oct 30 21:58:21.058382 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:21.057019    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls podName:9e1c052f-90b4-47c6-a636-dd77c751cba9 nodeName:}" failed. No retries permitted until 2026-10-30 21:58:21.556999793 +0000 UTC m=+1449.834693262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls") pod "maas-api-e2e-ait-be51942b-67454db999-j5vpx" (UID: "9e1c052f-90b4-47c6-a636-dd77c751cba9") : secret "maas-api-e2e-ait-be51942b-serving-cert" not found
Oct 30 21:58:21.169457 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.163435    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-metrics-certs\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.189385 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.187982    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9rjt\" (UniqueName: \"kubernetes.io/projected/9e1c052f-90b4-47c6-a636-dd77c751cba9-kube-api-access-v9rjt\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.258781 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.258744    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:21.259272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.259242    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:21.618849 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.618812    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.621161 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.621135    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls\") pod \"maas-api-e2e-ait-be51942b-67454db999-j5vpx\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:21.698749 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:21.698720    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:22.113684 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:22.113588    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"]
Oct 30 21:58:22.117288 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:58:22.117257    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1c052f_90b4_47c6_a636_dd77c751cba9.slice/crio-a03fa635cf26324125355d60fc88e41505acb0155e3689559527357fd5578c4d WatchSource:0}: Error finding container a03fa635cf26324125355d60fc88e41505acb0155e3689559527357fd5578c4d: Status 404 returned error can't find the container with id a03fa635cf26324125355d60fc88e41505acb0155e3689559527357fd5578c4d
Oct 30 21:58:22.263372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:22.262862    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:22.263372 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:22.262921    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:22.874874 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:22.874834    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx" event={"ID":"9e1c052f-90b4-47c6-a636-dd77c751cba9","Type":"ContainerStarted","Data":"a03fa635cf26324125355d60fc88e41505acb0155e3689559527357fd5578c4d"}
Oct 30 21:58:23.117664 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:23.117610    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:58:23.260106 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:23.259987    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:23.260106 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:23.260043    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:23.884198 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:23.884167    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx" event={"ID":"9e1c052f-90b4-47c6-a636-dd77c751cba9","Type":"ContainerStarted","Data":"bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d"}
Oct 30 21:58:23.884626 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:23.884230    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:24.250453 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:24.247847    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:24.250453 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:24.247928    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:25.246769 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:25.246732    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:58:25.247153 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:25.246783    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:58:26.619005 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:26.618970    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:26.619608 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:26.619541    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:26.621065 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:26.621044    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 21:58:26.851006 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:26.850947    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx" podStartSLOduration=5.866123082 podStartE2EDuration="6.850935329s" totalImagesPullingTime="984.812247ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:24.019981258 +0000 UTC m=+1452.297674730" watchObservedRunningTime="2026-10-30 21:58:26.850935329 +0000 UTC m=+1455.128628801"
Oct 30 21:58:29.898334 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:29.898303    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 21:58:35.846329 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:35.846242    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"]
Oct 30 21:58:35.874950 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:35.872848    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:35.958028 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:35.957988    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da80600-a902-41f2-847e-dbe0349d0c3c-openshift-service-ca\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:35.958659 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:35.958628    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8rww\" (UniqueName: \"kubernetes.io/projected/9da80600-a902-41f2-847e-dbe0349d0c3c-kube-api-access-p8rww\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:35.958928 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:35.958908    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:35.959090 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:35.959040    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-metrics-certs\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:35.989751 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:35.989718    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e585c53f-serving-cert\""
Oct 30 21:58:36.031741 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.031708    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"]
Oct 30 21:58:36.064990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.062689    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8rww\" (UniqueName: \"kubernetes.io/projected/9da80600-a902-41f2-847e-dbe0349d0c3c-kube-api-access-p8rww\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.064990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.062847    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.064990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.062895    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-metrics-certs\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.064990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.063072    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da80600-a902-41f2-847e-dbe0349d0c3c-openshift-service-ca\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.064990 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.063908    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da80600-a902-41f2-847e-dbe0349d0c3c-openshift-service-ca\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.064990 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:36.064277    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-serving-cert: secret "maas-api-e2e-ait-e585c53f-serving-cert" not found
Oct 30 21:58:36.064990 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:36.064362    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls podName:9da80600-a902-41f2-847e-dbe0349d0c3c nodeName:}" failed. No retries permitted until 2026-10-30 21:58:36.564328332 +0000 UTC m=+1464.842021784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls") pod "maas-api-e2e-ait-e585c53f-789474fb69-qmpjk" (UID: "9da80600-a902-41f2-847e-dbe0349d0c3c") : secret "maas-api-e2e-ait-e585c53f-serving-cert" not found
Oct 30 21:58:36.338374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.334126    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-metrics-certs\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.357398 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.354289    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8rww\" (UniqueName: \"kubernetes.io/projected/9da80600-a902-41f2-847e-dbe0349d0c3c-kube-api-access-p8rww\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.570387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.570224    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.581742 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.581697    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls\") pod \"maas-api-e2e-ait-e585c53f-789474fb69-qmpjk\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:36.857374 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:36.857020    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:37.275472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:37.275426    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.114496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:38.114451    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"]
Oct 30 21:58:39.126285 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:39.126249    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk" event={"ID":"9da80600-a902-41f2-847e-dbe0349d0c3c","Type":"ContainerStarted","Data":"f895b3ceffe47297bdd0ed3a3fa7d9b9c66289ab9947eba633ba07fc2d3460e4"}
Oct 30 21:58:40.135748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:40.135706    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk" event={"ID":"9da80600-a902-41f2-847e-dbe0349d0c3c","Type":"ContainerStarted","Data":"7b321108a6d5d29e49da54ee7f61fc92034989e8777b182e433b4e01193effa5"}
Oct 30 21:58:40.136123 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:40.135776    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:40.253277 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:40.253200    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk" podStartSLOduration=3.891369696 podStartE2EDuration="5.253180265s" totalImagesPullingTime="1.361810569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:40.252298858 +0000 UTC m=+1468.529992328" watchObservedRunningTime="2026-10-30 21:58:40.253180265 +0000 UTC m=+1468.530873739"
Oct 30 21:58:46.146752 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:58:46.146719    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 21:58:52.982220 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:58:52.982178    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:59:02.326614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.326555    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"]
Oct 30 21:59:02.331075 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.331051    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.375917 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.375883    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"]
Oct 30 21:59:02.380197 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.380179    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.426532 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.426495    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"]
Oct 30 21:59:02.438786 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.438746    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439057 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.438797    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439057 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.438944    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439057 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439031    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439232 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439188    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439322 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439302    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439429 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439340    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439490 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439429    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439547 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439513    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439713 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439661    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439802 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439719    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7vvx\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-kube-api-access-d7vvx\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439802 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439754    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.439802 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439777    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439936 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439803    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439936 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439832    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f212bdad-d633-4c1a-a4e5-182b295746c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439936 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439862    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq22m\" (UniqueName: \"kubernetes.io/projected/f212bdad-d633-4c1a-a4e5-182b295746c7-kube-api-access-bq22m\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.439936 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439922    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.440136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.439946    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.496520 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.496451    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"]
Oct 30 21:59:02.541969 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.541925    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.542141 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542016    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.542198 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542146    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.542249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542197    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7vvx\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-kube-api-access-d7vvx\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.542249 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542230    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.542389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542258    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542286    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542314    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f212bdad-d633-4c1a-a4e5-182b295746c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542389 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542343    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq22m\" (UniqueName: \"kubernetes.io/projected/f212bdad-d633-4c1a-a4e5-182b295746c7-kube-api-access-bq22m\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542428    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.542614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542452    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542512    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542614 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542573    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542836 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542662    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.542836 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542819    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.542949 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542902    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.543016 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542985    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.543407 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.542512    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.543778 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.543757    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f212bdad-d633-4c1a-a4e5-182b295746c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.544529 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.544505    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.544919 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.544896    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.545250 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.545222    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.545392 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.545372    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.545501 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.545338    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.545618 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.545603    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.545780 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.545755    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.545852 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.545807    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.545967 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.545942    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.546135 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.546103    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f212bdad-d633-4c1a-a4e5-182b295746c7-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.546267 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.546251    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.546420 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.546391    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.546746 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.546727    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.611452 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.611377    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f212bdad-d633-4c1a-a4e5-182b295746c7-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.612796 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.612769    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq22m\" (UniqueName: \"kubernetes.io/projected/f212bdad-d633-4c1a-a4e5-182b295746c7-kube-api-access-bq22m\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nvkls\" (UID: \"f212bdad-d633-4c1a-a4e5-182b295746c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.617387 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.617345    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.620895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.620874    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7vvx\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-kube-api-access-d7vvx\") pod \"maas-default-gateway-openshift-default-74854d8ccf-pxw88\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:02.645492 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.645472    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:02.695293 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:02.695255    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:03.111223 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.111168    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"]
Oct 30 21:59:03.111598 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:59:03.111573    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf212bdad_d633_4c1a_a4e5_182b295746c7.slice/crio-55fb60170109213704dd046413b269c1605bbdc0cb362b3ce9f654e5ba30d320 WatchSource:0}: Error finding container 55fb60170109213704dd046413b269c1605bbdc0cb362b3ce9f654e5ba30d320: Status 404 returned error can't find the container with id 55fb60170109213704dd046413b269c1605bbdc0cb362b3ce9f654e5ba30d320
Oct 30 21:59:03.114152 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.114115    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:03.114248 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.114180    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:03.114248 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.114209    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:03.114895 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.114875    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"]
Oct 30 21:59:03.147545 ip-10-0-135-39 kubenswrapper[5561]: W1030 21:59:03.147522    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7446d0_d8a5_4e3f_9895_49698bfa488d.slice/crio-e5390b0b1d52a3d77845bb085b9f359c4cdb327a17eb906eafbca15a2941ebe4 WatchSource:0}: Error finding container e5390b0b1d52a3d77845bb085b9f359c4cdb327a17eb906eafbca15a2941ebe4: Status 404 returned error can't find the container with id e5390b0b1d52a3d77845bb085b9f359c4cdb327a17eb906eafbca15a2941ebe4
Oct 30 21:59:03.150127 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.150094    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:03.150221 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.150162    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:03.150221 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.150204    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:03.323171 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.323128    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" event={"ID":"f212bdad-d633-4c1a-a4e5-182b295746c7","Type":"ContainerStarted","Data":"e47ed87a8b05e4a6544038593b8ab981ab77146d326e3a7aa918007dc35091aa"}
Oct 30 21:59:03.323412 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.323225    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" event={"ID":"f212bdad-d633-4c1a-a4e5-182b295746c7","Type":"ContainerStarted","Data":"55fb60170109213704dd046413b269c1605bbdc0cb362b3ce9f654e5ba30d320"}
Oct 30 21:59:03.325522 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.325497    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" event={"ID":"ce7446d0-d8a5-4e3f-9895-49698bfa488d","Type":"ContainerStarted","Data":"1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303"}
Oct 30 21:59:03.325679 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.325667    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" event={"ID":"ce7446d0-d8a5-4e3f-9895-49698bfa488d","Type":"ContainerStarted","Data":"e5390b0b1d52a3d77845bb085b9f359c4cdb327a17eb906eafbca15a2941ebe4"}
Oct 30 21:59:03.459104 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.459045    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podStartSLOduration=1.459033216 podStartE2EDuration="1.459033216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:03.45868778 +0000 UTC m=+1491.736381262" watchObservedRunningTime="2026-10-30 21:59:03.459033216 +0000 UTC m=+1491.736726686"
Oct 30 21:59:03.601403 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.601326    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podStartSLOduration=1.601311311 podStartE2EDuration="1.601311311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:03.599236059 +0000 UTC m=+1491.876929533" watchObservedRunningTime="2026-10-30 21:59:03.601311311 +0000 UTC m=+1491.879004782"
Oct 30 21:59:03.646457 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.646415    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:03.697414 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:03.695593    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:04.652496 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:04.652457    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:59:04.653111 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:04.653075    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:59:04.697143 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:04.697107    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:59:04.697343 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:04.697172    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:59:05.675376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:05.674581    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:59:05.675376 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:05.674652    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:59:05.696241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:05.696010    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:59:05.696241 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:05.696101    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:59:06.656373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:06.652725    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:59:06.656373 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:06.652789    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:59:06.696501 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:06.696460    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:59:06.696874 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:06.696542    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:59:07.657682 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:07.657645    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:59:07.657975 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:07.657944    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:59:07.696748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:07.696634    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:59:07.696748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:07.696693    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:59:08.647738 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:08.647696    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:59:08.648001 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:08.647972    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:59:08.696298 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:08.696088    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:59:08.696298 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:08.696164    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:59:09.646612 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:09.646578    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:59:09.646993 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:09.646636    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:59:09.696401 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:09.696368    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:59:09.696546 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:09.696420    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:59:10.646422 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:10.646387    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:59:10.646591 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:10.646439    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:59:10.695992 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:10.695961    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:59:10.696399 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:10.696018    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:59:12.177206 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.177177    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:59:12.178300 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.178274    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 21:59:12.215375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.214750    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:12.215821 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.215793    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:12.218023 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.217708    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls"
Oct 30 21:59:12.292501 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.292453    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:12.292929 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.292907    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:12.293803 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:12.293785    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 21:59:17.173996 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:17.173953    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 21:59:22.840497 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:22.840455    5561 patch_prober.go:29] interesting pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:59:22.842844 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:22.840513    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:23.000097 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:59:23.000063    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:59:23.616059 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:23.616019    5561 generic.go:356] "Generic (PLEG): container finished" podID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerID="23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53" exitCode=0
Oct 30 21:59:23.616272 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:23.616241    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" event={"ID":"4cdae184-f33e-4f49-9d68-85cc9aecb2b9","Type":"ContainerDied","Data":"23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53"}
Oct 30 21:59:23.616816 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:23.616793    5561 scope.go:122] "RemoveContainer" containerID="23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53"
Oct 30 21:59:23.618961 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:23.618924    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:23.619068 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:23.619014    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:23.619136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:23.619120    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:24.675416 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:24.670509    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" event={"ID":"4cdae184-f33e-4f49-9d68-85cc9aecb2b9","Type":"ContainerStarted","Data":"e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4"}
Oct 30 21:59:25.659375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:25.657694    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:59:25.659375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:25.657811    5561 patch_prober.go:29] interesting pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:59:25.659375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:25.657856    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 21:59:26.666541 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:26.666506    5561 patch_prober.go:29] interesting pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:59:26.667157 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:26.667115    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 21:59:27.669756 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:27.669713    5561 patch_prober.go:29] interesting pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:59:27.670157 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:27.669788    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 21:59:28.664439 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.663986    5561 patch_prober.go:29] interesting pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:59:28.664439 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.664056    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 21:59:28.701022 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.700995    5561 generic.go:356] "Generic (PLEG): container finished" podID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerID="383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8" exitCode=0
Oct 30 21:59:28.701375 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.701054    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" event={"ID":"0be698b7-5a83-4d30-8ace-917f0ac6fe1a","Type":"ContainerDied","Data":"383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8"}
Oct 30 21:59:28.701513 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.701496    5561 scope.go:122] "RemoveContainer" containerID="383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8"
Oct 30 21:59:28.703079 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.703046    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:28.703145 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.703129    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:28.703184 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:28.703165    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:59:29.656140 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:29.656096    5561 patch_prober.go:29] interesting pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:59:29.656321 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:29.656182    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 21:59:29.710766 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:29.710730    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" event={"ID":"0be698b7-5a83-4d30-8ace-917f0ac6fe1a","Type":"ContainerStarted","Data":"451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26"}
Oct 30 21:59:30.087947 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:30.087864    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:59:30.099313 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:30.099279    5561 patch_prober.go:29] interesting pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:59:30.099638 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:30.099599    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:59:30.673283 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:30.673245    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:59:30.673727 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:30.673705    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:59:30.674470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:30.674449    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 21:59:31.076434 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:31.075700    5561 patch_prober.go:29] interesting pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:59:31.076434 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:31.075759    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:59:31.400537 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:31.400499    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:59:32.073754 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:32.073719    5561 patch_prober.go:29] interesting pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:59:32.073922 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:32.073773    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:59:33.072909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:33.072871    5561 patch_prober.go:29] interesting pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:59:33.073422 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:33.072948    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:59:34.072854 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:34.072822    5561 patch_prober.go:29] interesting pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:59:34.072999 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:34.072891    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:59:35.072877 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:35.072839    5561 patch_prober.go:29] interesting pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:59:35.073062 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:35.072905    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:59:36.076748 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:36.076718    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:59:36.077709 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:36.077688    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 21:59:36.749128 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:36.749093    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"]
Oct 30 21:59:41.750305 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:41.750268    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x" podUID="1cd030ec-a67e-4958-908a-c475d14aea57" containerName="payload-pre-processing" containerID="cri-o://3da99acf1314a118cbf8aa676a737f15f81eed369dcfa00255c254b3cc6a8852" gracePeriod=25
Oct 30 21:59:41.910472 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:41.910433    5561 generic.go:356] "Generic (PLEG): container finished" podID="1cd030ec-a67e-4958-908a-c475d14aea57" containerID="3da99acf1314a118cbf8aa676a737f15f81eed369dcfa00255c254b3cc6a8852" exitCode=0
Oct 30 21:59:41.910626 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:41.910477    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x" event={"ID":"1cd030ec-a67e-4958-908a-c475d14aea57","Type":"ContainerDied","Data":"3da99acf1314a118cbf8aa676a737f15f81eed369dcfa00255c254b3cc6a8852"}
Oct 30 21:59:42.046919 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.046890    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:59:42.161779 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.161752    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cd030ec-a67e-4958-908a-c475d14aea57-kube-api-access-xcrhk\" (UniqueName: \"kubernetes.io/projected/1cd030ec-a67e-4958-908a-c475d14aea57-kube-api-access-xcrhk\") pod \"1cd030ec-a67e-4958-908a-c475d14aea57\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") "
Oct 30 21:59:42.161964 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.161830    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cd030ec-a67e-4958-908a-c475d14aea57-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd030ec-a67e-4958-908a-c475d14aea57-plugins-config-volume\") pod \"1cd030ec-a67e-4958-908a-c475d14aea57\" (UID: \"1cd030ec-a67e-4958-908a-c475d14aea57\") "
Oct 30 21:59:42.162266 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.162241    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd030ec-a67e-4958-908a-c475d14aea57-plugins-config-volume" pod "1cd030ec-a67e-4958-908a-c475d14aea57" (UID: "1cd030ec-a67e-4958-908a-c475d14aea57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:42.162719 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.162701    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cd030ec-a67e-4958-908a-c475d14aea57-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:42.163909 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.163886    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd030ec-a67e-4958-908a-c475d14aea57-kube-api-access-xcrhk" pod "1cd030ec-a67e-4958-908a-c475d14aea57" (UID: "1cd030ec-a67e-4958-908a-c475d14aea57"). InnerVolumeSpecName "kube-api-access-xcrhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:42.263807 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.263727    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcrhk\" (UniqueName: \"kubernetes.io/projected/1cd030ec-a67e-4958-908a-c475d14aea57-kube-api-access-xcrhk\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:42.918423 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.918384    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x" event={"ID":"1cd030ec-a67e-4958-908a-c475d14aea57","Type":"ContainerDied","Data":"3ba8740e0ace040276abe2086ee928b0dace21922fb7d63fccddc0fe4a905c05"}
Oct 30 21:59:42.918849 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.918427    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"
Oct 30 21:59:42.918849 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:42.918455    5561 scope.go:122] "RemoveContainer" containerID="3da99acf1314a118cbf8aa676a737f15f81eed369dcfa00255c254b3cc6a8852"
Oct 30 21:59:43.094022 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:43.093987    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"]
Oct 30 21:59:43.100175 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:43.100150    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x"]
Oct 30 21:59:43.848099 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:43.848068    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cd030ec-a67e-4958-908a-c475d14aea57" path="/var/lib/kubelet/pods/1cd030ec-a67e-4958-908a-c475d14aea57/volumes"
Oct 30 21:59:51.808530 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:51.808497    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"]
Oct 30 21:59:52.982428 ip-10-0-135-39 kubenswrapper[5561]: E1030 21:59:52.982390    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 21:59:56.809136 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:56.809068    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh" podUID="9c77946a-7423-44c7-855f-45c34ee9ddb8" containerName="payload-processing" containerID="cri-o://1ee2b9f28e6d153d041c64e3a99d9ecc2b3b498b3545a3c962bac3d6d4df2050" gracePeriod=25
Oct 30 21:59:57.039636 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.039598    5561 generic.go:356] "Generic (PLEG): container finished" podID="9c77946a-7423-44c7-855f-45c34ee9ddb8" containerID="1ee2b9f28e6d153d041c64e3a99d9ecc2b3b498b3545a3c962bac3d6d4df2050" exitCode=0
Oct 30 21:59:57.039982 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.039675    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh" event={"ID":"9c77946a-7423-44c7-855f-45c34ee9ddb8","Type":"ContainerDied","Data":"1ee2b9f28e6d153d041c64e3a99d9ecc2b3b498b3545a3c962bac3d6d4df2050"}
Oct 30 21:59:57.104935 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.104910    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:59:57.192258 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.192225    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c77946a-7423-44c7-855f-45c34ee9ddb8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77946a-7423-44c7-855f-45c34ee9ddb8-plugins-config-volume\") pod \"9c77946a-7423-44c7-855f-45c34ee9ddb8\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") "
Oct 30 21:59:57.192461 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.192266    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c77946a-7423-44c7-855f-45c34ee9ddb8-kube-api-access-g5fjj\" (UniqueName: \"kubernetes.io/projected/9c77946a-7423-44c7-855f-45c34ee9ddb8-kube-api-access-g5fjj\") pod \"9c77946a-7423-44c7-855f-45c34ee9ddb8\" (UID: \"9c77946a-7423-44c7-855f-45c34ee9ddb8\") "
Oct 30 21:59:57.192700 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.192676    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c77946a-7423-44c7-855f-45c34ee9ddb8-plugins-config-volume" pod "9c77946a-7423-44c7-855f-45c34ee9ddb8" (UID: "9c77946a-7423-44c7-855f-45c34ee9ddb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:57.193082 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.193062    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77946a-7423-44c7-855f-45c34ee9ddb8-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:57.194470 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.194442    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c77946a-7423-44c7-855f-45c34ee9ddb8-kube-api-access-g5fjj" pod "9c77946a-7423-44c7-855f-45c34ee9ddb8" (UID: "9c77946a-7423-44c7-855f-45c34ee9ddb8"). InnerVolumeSpecName "kube-api-access-g5fjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:57.294841 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:57.294807    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5fjj\" (UniqueName: \"kubernetes.io/projected/9c77946a-7423-44c7-855f-45c34ee9ddb8-kube-api-access-g5fjj\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:59:58.048526 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:58.048483    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"
Oct 30 21:59:58.048526 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:58.048510    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh" event={"ID":"9c77946a-7423-44c7-855f-45c34ee9ddb8","Type":"ContainerDied","Data":"191a6558f0b606259ccb1ee83596e0d1a16cefe593faf2b9d35fbeb7cce9ce51"}
Oct 30 21:59:58.049089 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:58.048582    5561 scope.go:122] "RemoveContainer" containerID="1ee2b9f28e6d153d041c64e3a99d9ecc2b3b498b3545a3c962bac3d6d4df2050"
Oct 30 21:59:58.229437 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:58.229396    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"]
Oct 30 21:59:58.283088 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:58.283048    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh"]
Oct 30 21:59:59.847209 ip-10-0-135-39 kubenswrapper[5561]: I1030 21:59:59.847173    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c77946a-7423-44c7-855f-45c34ee9ddb8" path="/var/lib/kubelet/pods/9c77946a-7423-44c7-855f-45c34ee9ddb8/volumes"
Oct 30 22:00:05.721127 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:05.721088    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"]
Oct 30 22:00:05.721540 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:05.721305    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf" podUID="cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" containerName="maas-api" containerID="cri-o://b0dbb1ccce264a35156489a57b61daf1350512a95a8079849d34312d18e22e55" gracePeriod=30
Oct 30 22:00:06.313867 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.313831    5561 generic.go:356] "Generic (PLEG): container finished" podID="cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" containerID="b0dbb1ccce264a35156489a57b61daf1350512a95a8079849d34312d18e22e55" exitCode=0
Oct 30 22:00:06.314406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.314375    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf" event={"ID":"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8","Type":"ContainerDied","Data":"b0dbb1ccce264a35156489a57b61daf1350512a95a8079849d34312d18e22e55"}
Oct 30 22:00:06.314639 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.314606    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf" event={"ID":"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8","Type":"ContainerDied","Data":"df24222f247f174f3c2b47e6f515496253191827a4cd2b8d067484faf3e257b5"}
Oct 30 22:00:06.314796 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.314779    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df24222f247f174f3c2b47e6f515496253191827a4cd2b8d067484faf3e257b5"
Oct 30 22:00:06.315106 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.315088    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 22:00:06.402190 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.402157    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-kube-api-access-vwpzv\" (UniqueName: \"kubernetes.io/projected/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-kube-api-access-vwpzv\") pod \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") "
Oct 30 22:00:06.402917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.402885    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-openshift-service-ca\") pod \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") "
Oct 30 22:00:06.403324 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.403297    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-metrics-certs\") pod \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") "
Oct 30 22:00:06.403659 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.403642    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls\") pod \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\" (UID: \"cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8\") "
Oct 30 22:00:06.405316 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.405294    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-openshift-service-ca" pod "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" (UID: "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:06.434879 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.434842    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-metrics-certs" pod "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" (UID: "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:06.435294 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.435258    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls" pod "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" (UID: "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:06.462892 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.451906    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-kube-api-access-vwpzv" pod "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" (UID: "cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8"). InnerVolumeSpecName "kube-api-access-vwpzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:06.505900 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.505864    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwpzv\" (UniqueName: \"kubernetes.io/projected/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-kube-api-access-vwpzv\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:06.505900 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.505900    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:06.506137 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.505914    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:06.506137 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:06.505927    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:07.321721 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:07.321690    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"
Oct 30 22:00:07.505870 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:07.505839    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"]
Oct 30 22:00:07.535956 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:07.535917    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f05b6443-665cd7774c-k4fcf"]
Oct 30 22:00:07.847645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:07.847611    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" path="/var/lib/kubelet/pods/cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8/volumes"
Oct 30 22:00:08.984848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:08.984815    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"]
Oct 30 22:00:08.985335 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:08.985091    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy" containerID="cri-o://e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4" gracePeriod=30
Oct 30 22:00:10.540180 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:10.540147    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"]
Oct 30 22:00:10.738571 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:10.738539    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"]
Oct 30 22:00:13.514593 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:13.514562    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"]
Oct 30 22:00:13.514983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:13.514759    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx" podUID="9e1c052f-90b4-47c6-a636-dd77c751cba9" containerName="maas-api" containerID="cri-o://bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d" gracePeriod=30
Oct 30 22:00:13.994403 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:13.977531    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 22:00:14.143459 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.143418    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls\") pod \"9e1c052f-90b4-47c6-a636-dd77c751cba9\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") "
Oct 30 22:00:14.143887 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.143861    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-metrics-certs\") pod \"9e1c052f-90b4-47c6-a636-dd77c751cba9\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") "
Oct 30 22:00:14.144194 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.144175    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e1c052f-90b4-47c6-a636-dd77c751cba9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1c052f-90b4-47c6-a636-dd77c751cba9-openshift-service-ca\") pod \"9e1c052f-90b4-47c6-a636-dd77c751cba9\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") "
Oct 30 22:00:14.144512 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.144490    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e1c052f-90b4-47c6-a636-dd77c751cba9-kube-api-access-v9rjt\" (UniqueName: \"kubernetes.io/projected/9e1c052f-90b4-47c6-a636-dd77c751cba9-kube-api-access-v9rjt\") pod \"9e1c052f-90b4-47c6-a636-dd77c751cba9\" (UID: \"9e1c052f-90b4-47c6-a636-dd77c751cba9\") "
Oct 30 22:00:14.145625 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.145596    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1c052f-90b4-47c6-a636-dd77c751cba9-openshift-service-ca" pod "9e1c052f-90b4-47c6-a636-dd77c751cba9" (UID: "9e1c052f-90b4-47c6-a636-dd77c751cba9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:14.185384 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.182569    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-metrics-certs" pod "9e1c052f-90b4-47c6-a636-dd77c751cba9" (UID: "9e1c052f-90b4-47c6-a636-dd77c751cba9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:14.185384 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.184412    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls" pod "9e1c052f-90b4-47c6-a636-dd77c751cba9" (UID: "9e1c052f-90b4-47c6-a636-dd77c751cba9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:14.235779 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.235666    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e1c052f-90b4-47c6-a636-dd77c751cba9-kube-api-access-v9rjt" pod "9e1c052f-90b4-47c6-a636-dd77c751cba9" (UID: "9e1c052f-90b4-47c6-a636-dd77c751cba9"). InnerVolumeSpecName "kube-api-access-v9rjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:14.287109 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.286903    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.287109 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.286938    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e1c052f-90b4-47c6-a636-dd77c751cba9-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.287109 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.286953    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1c052f-90b4-47c6-a636-dd77c751cba9-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.287109 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.286968    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9rjt\" (UniqueName: \"kubernetes.io/projected/9e1c052f-90b4-47c6-a636-dd77c751cba9-kube-api-access-v9rjt\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.440606 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.440573    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 22:00:14.476484 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.476452    5561 generic.go:356] "Generic (PLEG): container finished" podID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerID="e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4" exitCode=0
Oct 30 22:00:14.476644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.476487    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" event={"ID":"4cdae184-f33e-4f49-9d68-85cc9aecb2b9","Type":"ContainerDied","Data":"e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4"}
Oct 30 22:00:14.476644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.476501    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"
Oct 30 22:00:14.476644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.476543    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x" event={"ID":"4cdae184-f33e-4f49-9d68-85cc9aecb2b9","Type":"ContainerDied","Data":"5152fbedb4b66d37e20546c2a3bf0deabc9562badfa8d2378a74a4efb70fb813"}
Oct 30 22:00:14.476644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.476571    5561 scope.go:122] "RemoveContainer" containerID="e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4"
Oct 30 22:00:14.478001 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.477979    5561 generic.go:356] "Generic (PLEG): container finished" podID="9e1c052f-90b4-47c6-a636-dd77c751cba9" containerID="bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d" exitCode=0
Oct 30 22:00:14.478100 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.478019    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"
Oct 30 22:00:14.478100 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.478030    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx" event={"ID":"9e1c052f-90b4-47c6-a636-dd77c751cba9","Type":"ContainerDied","Data":"bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d"}
Oct 30 22:00:14.478100 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.478072    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx" event={"ID":"9e1c052f-90b4-47c6-a636-dd77c751cba9","Type":"ContainerDied","Data":"a03fa635cf26324125355d60fc88e41505acb0155e3689559527357fd5578c4d"}
Oct 30 22:00:14.487024 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.487008    5561 scope.go:122] "RemoveContainer" containerID="23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53"
Oct 30 22:00:14.489340 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489317    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-credential-socket\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.489455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489410    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-podinfo\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.489499 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489464    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-kube-api-access-fhr7s\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-kube-api-access-fhr7s\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.489539 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489524    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-credential-socket" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:14.489581 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489552    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-certs\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.489623 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489609    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-data\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.489668 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489660    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istiod-ca-cert\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.489800 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489782    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-envoy\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.489875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489802    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-data" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:14.489875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489814    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-certs" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:14.489875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489859    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-socket\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.490032 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.489936    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-token\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-token\") pod \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\" (UID: \"4cdae184-f33e-4f49-9d68-85cc9aecb2b9\") "
Oct 30 22:00:14.490083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.490021    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istiod-ca-cert" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:14.490450 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.490147    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-socket" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:14.490697 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.490678    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.490775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.490703    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.490775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.490716    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.490775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.490729    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.490775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.490743    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.492120 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.492096    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-podinfo" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:00:14.492201 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.492137    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-kube-api-access-fhr7s" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "kube-api-access-fhr7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:14.492380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.492327    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-envoy" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:14.492481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.492462    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-token" pod "4cdae184-f33e-4f49-9d68-85cc9aecb2b9" (UID: "4cdae184-f33e-4f49-9d68-85cc9aecb2b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:14.497457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.497438    5561 scope.go:122] "RemoveContainer" containerID="e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4"
Oct 30 22:00:14.497864 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:14.497843    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4\": container with ID starting with e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4 not found: ID does not exist" containerID="e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4"
Oct 30 22:00:14.497923 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.497871    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4"} err="failed to get container status \"e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4\": rpc error: code = NotFound desc = could not find container \"e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4\": container with ID starting with e2f6b9eb7f76e468371d5b8a1d17fabba66327fbaf3ae9827eef391e37207aa4 not found: ID does not exist"
Oct 30 22:00:14.497923 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.497887    5561 scope.go:122] "RemoveContainer" containerID="23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53"
Oct 30 22:00:14.498170 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:14.498147    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53\": container with ID starting with 23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53 not found: ID does not exist" containerID="23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53"
Oct 30 22:00:14.498216 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.498180    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53"} err="failed to get container status \"23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53\": rpc error: code = NotFound desc = could not find container \"23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53\": container with ID starting with 23cd5bc1955eb3fa2c471a6bd0295dd4ca700b1da814dfe5170a1b5ab15b8b53 not found: ID does not exist"
Oct 30 22:00:14.498216 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.498203    5561 scope.go:122] "RemoveContainer" containerID="bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d"
Oct 30 22:00:14.507115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.507100    5561 scope.go:122] "RemoveContainer" containerID="bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d"
Oct 30 22:00:14.507422 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:14.507398    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d\": container with ID starting with bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d not found: ID does not exist" containerID="bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d"
Oct 30 22:00:14.507485 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.507427    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d"} err="failed to get container status \"bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d\": rpc error: code = NotFound desc = could not find container \"bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d\": container with ID starting with bbebd93993ec3df8689fda70b1fca8556786e2db5332dac5104daa673c94583d not found: ID does not exist"
Oct 30 22:00:14.762757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.633317    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.762757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.633381    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.762757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.633407    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.762757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.633422    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhr7s\" (UniqueName: \"kubernetes.io/projected/4cdae184-f33e-4f49-9d68-85cc9aecb2b9-kube-api-access-fhr7s\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.966914 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:14.966879    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"]
Oct 30 22:00:15.016536 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.016460    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be51942b-67454db999-j5vpx"]
Oct 30 22:00:15.086848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.086811    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"]
Oct 30 22:00:15.094129 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.094098    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x"]
Oct 30 22:00:15.541509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.541452    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v" podUID="3f7400ac-d88c-4cf2-ace8-2ac12d762714" containerName="payload-processing" containerID="cri-o://ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f" gracePeriod=25
Oct 30 22:00:15.753305 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.751836    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm" podUID="af485eae-fedb-41c6-bb72-7c6ea2d4cc19" containerName="payload-pre-processing" containerID="cri-o://4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb" gracePeriod=25
Oct 30 22:00:15.906420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.902392    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" path="/var/lib/kubelet/pods/4cdae184-f33e-4f49-9d68-85cc9aecb2b9/volumes"
Oct 30 22:00:15.906420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.903099    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e1c052f-90b4-47c6-a636-dd77c751cba9" path="/var/lib/kubelet/pods/9e1c052f-90b4-47c6-a636-dd77c751cba9/volumes"
Oct 30 22:00:15.954222 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:15.954197    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 22:00:16.007127 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.007094    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f7400ac-d88c-4cf2-ace8-2ac12d762714-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7400ac-d88c-4cf2-ace8-2ac12d762714-plugins-config-volume\") pod \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") "
Oct 30 22:00:16.007370 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.007338    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f7400ac-d88c-4cf2-ace8-2ac12d762714-kube-api-access-z5kkd\" (UniqueName: \"kubernetes.io/projected/3f7400ac-d88c-4cf2-ace8-2ac12d762714-kube-api-access-z5kkd\") pod \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\" (UID: \"3f7400ac-d88c-4cf2-ace8-2ac12d762714\") "
Oct 30 22:00:16.007621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.007590    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f7400ac-d88c-4cf2-ace8-2ac12d762714-plugins-config-volume" pod "3f7400ac-d88c-4cf2-ace8-2ac12d762714" (UID: "3f7400ac-d88c-4cf2-ace8-2ac12d762714"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:16.008103 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.008079    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7400ac-d88c-4cf2-ace8-2ac12d762714-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:16.009738 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.009713    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f7400ac-d88c-4cf2-ace8-2ac12d762714-kube-api-access-z5kkd" pod "3f7400ac-d88c-4cf2-ace8-2ac12d762714" (UID: "3f7400ac-d88c-4cf2-ace8-2ac12d762714"). InnerVolumeSpecName "kube-api-access-z5kkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:16.088466 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.088434    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 22:00:16.109407 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.109377    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-plugins-config-volume\") pod \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") "
Oct 30 22:00:16.109563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.109453    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-kube-api-access-pfl4p\" (UniqueName: \"kubernetes.io/projected/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-kube-api-access-pfl4p\") pod \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\" (UID: \"af485eae-fedb-41c6-bb72-7c6ea2d4cc19\") "
Oct 30 22:00:16.109793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.109768    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-plugins-config-volume" pod "af485eae-fedb-41c6-bb72-7c6ea2d4cc19" (UID: "af485eae-fedb-41c6-bb72-7c6ea2d4cc19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:16.110227 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.110204    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5kkd\" (UniqueName: \"kubernetes.io/projected/3f7400ac-d88c-4cf2-ace8-2ac12d762714-kube-api-access-z5kkd\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:16.110326 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.110231    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:16.111706 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.111674    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-kube-api-access-pfl4p" pod "af485eae-fedb-41c6-bb72-7c6ea2d4cc19" (UID: "af485eae-fedb-41c6-bb72-7c6ea2d4cc19"). InnerVolumeSpecName "kube-api-access-pfl4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:16.212209 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.212019    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfl4p\" (UniqueName: \"kubernetes.io/projected/af485eae-fedb-41c6-bb72-7c6ea2d4cc19-kube-api-access-pfl4p\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:16.494414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.494342    5561 generic.go:356] "Generic (PLEG): container finished" podID="af485eae-fedb-41c6-bb72-7c6ea2d4cc19" containerID="4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb" exitCode=0
Oct 30 22:00:16.494414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.494395    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm" event={"ID":"af485eae-fedb-41c6-bb72-7c6ea2d4cc19","Type":"ContainerDied","Data":"4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb"}
Oct 30 22:00:16.494414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.494417    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"
Oct 30 22:00:16.494708 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.494462    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm" event={"ID":"af485eae-fedb-41c6-bb72-7c6ea2d4cc19","Type":"ContainerDied","Data":"7e4cb365224689ba21ed33ceaf9cbdb664fa60a40432fe4fac03a67cd9c79eef"}
Oct 30 22:00:16.494708 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.494492    5561 scope.go:122] "RemoveContainer" containerID="4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb"
Oct 30 22:00:16.496112 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.496088    5561 generic.go:356] "Generic (PLEG): container finished" podID="3f7400ac-d88c-4cf2-ace8-2ac12d762714" containerID="ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f" exitCode=0
Oct 30 22:00:16.496229 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.496149    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"
Oct 30 22:00:16.496229 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.496168    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v" event={"ID":"3f7400ac-d88c-4cf2-ace8-2ac12d762714","Type":"ContainerDied","Data":"ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f"}
Oct 30 22:00:16.496229 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.496205    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v" event={"ID":"3f7400ac-d88c-4cf2-ace8-2ac12d762714","Type":"ContainerDied","Data":"379b0c5d367761a87054d60f717bcbf9c8fc1a68da4be286d185826a5eb9de92"}
Oct 30 22:00:16.506877 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.506855    5561 scope.go:122] "RemoveContainer" containerID="4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb"
Oct 30 22:00:16.507165 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:16.507146    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb\": container with ID starting with 4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb not found: ID does not exist" containerID="4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb"
Oct 30 22:00:16.507233 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.507173    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb"} err="failed to get container status \"4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb\": rpc error: code = NotFound desc = could not find container \"4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb\": container with ID starting with 4a85d7c944e610819d6f87ad15a561bcf7516d8321887ddad2bd931c8d0910bb not found: ID does not exist"
Oct 30 22:00:16.507233 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.507200    5561 scope.go:122] "RemoveContainer" containerID="ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f"
Oct 30 22:00:16.517292 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.517260    5561 scope.go:122] "RemoveContainer" containerID="ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f"
Oct 30 22:00:16.517793 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:16.517762    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f\": container with ID starting with ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f not found: ID does not exist" containerID="ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f"
Oct 30 22:00:16.517911 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.517801    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f"} err="failed to get container status \"ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f\": rpc error: code = NotFound desc = could not find container \"ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f\": container with ID starting with ddd65a1f3a5e7da000372b7a3c6f8f42ca82d75b2327022cf9fb0f7560bc242f not found: ID does not exist"
Oct 30 22:00:16.644448 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.644416    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"]
Oct 30 22:00:16.644677 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.644640    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" containerID="cri-o://451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26" gracePeriod=30
Oct 30 22:00:16.685307 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.685277    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"]
Oct 30 22:00:16.695947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.695922    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm"]
Oct 30 22:00:16.905384 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.904435    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"]
Oct 30 22:00:16.909891 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:16.909859    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v"]
Oct 30 22:00:17.849321 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:17.849279    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f7400ac-d88c-4cf2-ace8-2ac12d762714" path="/var/lib/kubelet/pods/3f7400ac-d88c-4cf2-ace8-2ac12d762714/volumes"
Oct 30 22:00:17.849939 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:17.849921    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af485eae-fedb-41c6-bb72-7c6ea2d4cc19" path="/var/lib/kubelet/pods/af485eae-fedb-41c6-bb72-7c6ea2d4cc19/volumes"
Oct 30 22:00:18.810341 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.810294    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"]
Oct 30 22:00:18.811310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811288    5561 cpu_manager.go:414] "containerMap: removing container" podUID="af485eae-fedb-41c6-bb72-7c6ea2d4cc19" containerName="payload-pre-processing"
Oct 30 22:00:18.811419 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811313    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af485eae-fedb-41c6-bb72-7c6ea2d4cc19" containerName="payload-pre-processing"
Oct 30 22:00:18.811419 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811371    5561 cpu_manager.go:414] "containerMap: removing container" podUID="cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" containerName="maas-api"
Oct 30 22:00:18.811419 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811380    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" containerName="maas-api"
Oct 30 22:00:18.811419 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811396    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9c77946a-7423-44c7-855f-45c34ee9ddb8" containerName="payload-processing"
Oct 30 22:00:18.811419 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811403    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c77946a-7423-44c7-855f-45c34ee9ddb8" containerName="payload-processing"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811432    5561 cpu_manager.go:414] "containerMap: removing container" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811440    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811472    5561 cpu_manager.go:414] "containerMap: removing container" podUID="1cd030ec-a67e-4958-908a-c475d14aea57" containerName="payload-pre-processing"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811480    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cd030ec-a67e-4958-908a-c475d14aea57" containerName="payload-pre-processing"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811497    5561 cpu_manager.go:414] "containerMap: removing container" podUID="3f7400ac-d88c-4cf2-ace8-2ac12d762714" containerName="payload-processing"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811505    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f7400ac-d88c-4cf2-ace8-2ac12d762714" containerName="payload-processing"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811518    5561 cpu_manager.go:414] "containerMap: removing container" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811525    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811537    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9e1c052f-90b4-47c6-a636-dd77c751cba9" containerName="maas-api"
Oct 30 22:00:18.811673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811545    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e1c052f-90b4-47c6-a636-dd77c751cba9" containerName="maas-api"
Oct 30 22:00:18.812118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811692    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cd030ec-a67e-4958-908a-c475d14aea57" containerName="payload-pre-processing"
Oct 30 22:00:18.812118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811707    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f7400ac-d88c-4cf2-ace8-2ac12d762714" containerName="payload-processing"
Oct 30 22:00:18.812118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811716    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy"
Oct 30 22:00:18.812118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811726    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e1c052f-90b4-47c6-a636-dd77c751cba9" containerName="maas-api"
Oct 30 22:00:18.812118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811736    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="af485eae-fedb-41c6-bb72-7c6ea2d4cc19" containerName="payload-pre-processing"
Oct 30 22:00:18.812118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811748    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="cdf424c1-3d9c-4fb6-b946-3cc46dd27ec8" containerName="maas-api"
Oct 30 22:00:18.812118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.811758    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c77946a-7423-44c7-855f-45c34ee9ddb8" containerName="payload-processing"
Oct 30 22:00:18.817608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.817587    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857196 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857163    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-envoy\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857341 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857271    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2hd\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-kube-api-access-cs2hd\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857460    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-socket\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857515 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857497    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-credential-socket\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857553 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857526    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70638018-48d1-4ef3-b021-b974ee34d87a-istiod-ca-cert\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857595    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-certs\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857635    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-istio-token\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857689    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70638018-48d1-4ef3-b021-b974ee34d87a-istio-podinfo\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.857790 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.857778    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-data\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.923414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.923380    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2b8311b7-openshift-default-dockercfg-rzbd9\""
Oct 30 22:00:18.927284 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.927255    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"]
Oct 30 22:00:18.959331 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959295    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-envoy\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959512 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959409    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2hd\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-kube-api-access-cs2hd\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959590 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959577    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-socket\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959623    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-credential-socket\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959715 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959664    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70638018-48d1-4ef3-b021-b974ee34d87a-istiod-ca-cert\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959715 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959700    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-certs\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959816 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959749    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-istio-token\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959867 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959822    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70638018-48d1-4ef3-b021-b974ee34d87a-istio-podinfo\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.959965 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959948    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-data\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.960036 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.959959    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-socket\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.960036 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.960008    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-credential-socket\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.960170 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.960143    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-certs\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.960413 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.960383    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-data\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.960585 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.960562    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70638018-48d1-4ef3-b021-b974ee34d87a-istiod-ca-cert\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.962080 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.962059    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-envoy\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:18.962522 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:18.962491    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70638018-48d1-4ef3-b021-b974ee34d87a-istio-podinfo\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:19.080424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:19.076610    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2hd\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-kube-api-access-cs2hd\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:19.127383 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:19.124974    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-istio-token\") pod \"e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") " pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:19.154907 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:19.153571    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:19.662213 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:00:19.662164    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70638018_48d1_4ef3_b021_b974ee34d87a.slice/crio-7efc8db1ab32e32a51687c6f849757a0e9b4dc520eb2baa5de1b2fa0ff613ce8 WatchSource:0}: Error finding container 7efc8db1ab32e32a51687c6f849757a0e9b4dc520eb2baa5de1b2fa0ff613ce8: Status 404 returned error can't find the container with id 7efc8db1ab32e32a51687c6f849757a0e9b4dc520eb2baa5de1b2fa0ff613ce8
Oct 30 22:00:19.662412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:19.662389    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"]
Oct 30 22:00:19.664806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:19.664774    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:19.664870 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:19.664838    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:19.664914 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:19.664870    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:20.578113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:20.575539    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" event={"ID":"70638018-48d1-4ef3-b021-b974ee34d87a","Type":"ContainerStarted","Data":"77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e"}
Oct 30 22:00:20.578113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:20.575683    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" event={"ID":"70638018-48d1-4ef3-b021-b974ee34d87a","Type":"ContainerStarted","Data":"7efc8db1ab32e32a51687c6f849757a0e9b4dc520eb2baa5de1b2fa0ff613ce8"}
Oct 30 22:00:20.809372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:20.804068    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podStartSLOduration=2.804050632 podStartE2EDuration="2.804050632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:20.789094882 +0000 UTC m=+1569.066788357" watchObservedRunningTime="2026-10-30 22:00:20.804050632 +0000 UTC m=+1569.081744106"
Oct 30 22:00:21.000288 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.000253    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"]
Oct 30 22:00:21.001149 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.001125    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cdae184-f33e-4f49-9d68-85cc9aecb2b9" containerName="istio-proxy"
Oct 30 22:00:21.004770 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.004747    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.149221 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.144431    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-87e2c278-gw-openshift-default-dockercfg-z762v\""
Oct 30 22:00:21.171372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.151927    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"]
Oct 30 22:00:21.171372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.155806    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:21.194373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.189712    5561 patch_prober.go:29] interesting pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 30 22:00:21.194373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.189786    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:00:21.213383 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.211808    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:21.253343 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253057    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2glnp\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-kube-api-access-2glnp\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253343 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253116    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb83f97-7336-40d1-a749-511d9118b63d-istio-podinfo\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253343 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253183    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-certs\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253343 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253302    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb83f97-7336-40d1-a749-511d9118b63d-istiod-ca-cert\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253432    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-istio-token\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253465    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-credential-socket\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253524    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-socket\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253580    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-envoy\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.253681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.253638    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-data\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.361488    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-istio-token\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.361998    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-credential-socket\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.361556    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-credential-socket\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.362420    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-socket\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.362485    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-envoy\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.362551    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-data\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.362661    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2glnp\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-kube-api-access-2glnp\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.362694    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb83f97-7336-40d1-a749-511d9118b63d-istio-podinfo\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.362764    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-certs\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.369421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.362909    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb83f97-7336-40d1-a749-511d9118b63d-istiod-ca-cert\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.395652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.395335    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-socket\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.396624 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.395894    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb83f97-7336-40d1-a749-511d9118b63d-istiod-ca-cert\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.396624 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.396017    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-data\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.396624 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.396247    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-certs\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.484889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.483085    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2glnp\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-kube-api-access-2glnp\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.502392 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.501790    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-envoy\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.503570 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.503434    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-istio-token\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.516059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.515977    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb83f97-7336-40d1-a749-511d9118b63d-istio-podinfo\") pod \"e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") " pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:21.585123 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.585092    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:21.585970 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.585951    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:00:21.642527 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:21.642500    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:22.098586 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.098549    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"]
Oct 30 22:00:22.121200 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.121174    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 22:00:22.128991 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:00:22.128965    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb83f97_7336_40d1_a749_511d9118b63d.slice/crio-a5d91cb63ac0657ed2623305ac0b0794b8f4d0f638a6447ef4bad4783c4fb99a WatchSource:0}: Error finding container a5d91cb63ac0657ed2623305ac0b0794b8f4d0f638a6447ef4bad4783c4fb99a: Status 404 returned error can't find the container with id a5d91cb63ac0657ed2623305ac0b0794b8f4d0f638a6447ef4bad4783c4fb99a
Oct 30 22:00:22.131589 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.131557    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:22.131672 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.131633    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:22.131672 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.131665    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:22.253177 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253140    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-certs\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253198    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-data\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253313    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-envoy\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253512 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253418    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-token\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-token\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253512 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253455    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-podinfo\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253512 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253476    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-certs" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:22.253649 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253618    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istiod-ca-cert\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253673    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-credential-socket\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253751 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253729    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-kube-api-access-zckhw\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-kube-api-access-zckhw\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.253804 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.253767    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-socket\") pod \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\" (UID: \"0be698b7-5a83-4d30-8ace-917f0ac6fe1a\") "
Oct 30 22:00:22.254999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.254635    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-data" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:22.254999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.254727    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.254999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.254910    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-socket" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:22.254999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.254951    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-credential-socket" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:22.254999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.254982    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istiod-ca-cert" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:22.256292 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.256265    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-envoy" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:22.256658 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.256632    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-podinfo" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:00:22.257110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.257089    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-kube-api-access-zckhw" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "kube-api-access-zckhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:22.257475 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.257453    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-token" pod "0be698b7-5a83-4d30-8ace-917f0ac6fe1a" (UID: "0be698b7-5a83-4d30-8ace-917f0ac6fe1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:22.356794 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356761    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.356794 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356789    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.356794 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356798    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zckhw\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-kube-api-access-zckhw\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.357084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356810    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.357084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356818    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.357084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356826    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.357084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356835    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.357084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.356843    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be698b7-5a83-4d30-8ace-917f0ac6fe1a-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:22.593603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.593568    5561 generic.go:356] "Generic (PLEG): container finished" podID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerID="451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26" exitCode=0
Oct 30 22:00:22.594086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.593618    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"
Oct 30 22:00:22.594086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.593649    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" event={"ID":"0be698b7-5a83-4d30-8ace-917f0ac6fe1a","Type":"ContainerDied","Data":"451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26"}
Oct 30 22:00:22.594086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.593717    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2" event={"ID":"0be698b7-5a83-4d30-8ace-917f0ac6fe1a","Type":"ContainerDied","Data":"6030e4e49b07185bdf2727fcec8318dfa92d010df68caa7ab00e588766b2e17b"}
Oct 30 22:00:22.594086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.593749    5561 scope.go:122] "RemoveContainer" containerID="451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26"
Oct 30 22:00:22.595538 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.595469    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" event={"ID":"cfb83f97-7336-40d1-a749-511d9118b63d","Type":"ContainerStarted","Data":"f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3"}
Oct 30 22:00:22.595652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.595551    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" event={"ID":"cfb83f97-7336-40d1-a749-511d9118b63d","Type":"ContainerStarted","Data":"a5d91cb63ac0657ed2623305ac0b0794b8f4d0f638a6447ef4bad4783c4fb99a"}
Oct 30 22:00:22.604461 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.604441    5561 scope.go:122] "RemoveContainer" containerID="383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8"
Oct 30 22:00:22.615563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.615544    5561 scope.go:122] "RemoveContainer" containerID="451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26"
Oct 30 22:00:22.615863 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:22.615842    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26\": container with ID starting with 451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26 not found: ID does not exist" containerID="451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26"
Oct 30 22:00:22.615925 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.615878    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26"} err="failed to get container status \"451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26\": rpc error: code = NotFound desc = could not find container \"451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26\": container with ID starting with 451c83249e3148d29f5b35f14b69dc03477076f929794320425049d883963d26 not found: ID does not exist"
Oct 30 22:00:22.615925 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.615900    5561 scope.go:122] "RemoveContainer" containerID="383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8"
Oct 30 22:00:22.616150 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:22.616129    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8\": container with ID starting with 383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8 not found: ID does not exist" containerID="383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8"
Oct 30 22:00:22.616216 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.616157    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8"} err="failed to get container status \"383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8\": rpc error: code = NotFound desc = could not find container \"383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8\": container with ID starting with 383df7070faefc36a1bbbf3ab9ea3c05afcd82c48a4df1c317573f03b03c85c8 not found: ID does not exist"
Oct 30 22:00:22.643278 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.643258    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:22.764632 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.764556    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"]
Oct 30 22:00:22.813991 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.813964    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2"]
Oct 30 22:00:22.934869 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:22.934818    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podStartSLOduration=2.934806235 podStartE2EDuration="2.934806235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:22.932939467 +0000 UTC m=+1571.210632944" watchObservedRunningTime="2026-10-30 22:00:22.934806235 +0000 UTC m=+1571.212499706"
Oct 30 22:00:22.982688 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:22.982645    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:00:23.648131 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:23.648102    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:23.849009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:23.848969    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" path="/var/lib/kubelet/pods/0be698b7-5a83-4d30-8ace-917f0ac6fe1a/volumes"
Oct 30 22:00:24.610086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:24.610054    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:24.611028 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:24.611006    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:00:26.399421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.399393    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"]
Oct 30 22:00:26.399870 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.399628    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk" podUID="9da80600-a902-41f2-847e-dbe0349d0c3c" containerName="maas-api" containerID="cri-o://7b321108a6d5d29e49da54ee7f61fc92034989e8777b182e433b4e01193effa5" gracePeriod=30
Oct 30 22:00:26.627109 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.627077    5561 generic.go:356] "Generic (PLEG): container finished" podID="9da80600-a902-41f2-847e-dbe0349d0c3c" containerID="7b321108a6d5d29e49da54ee7f61fc92034989e8777b182e433b4e01193effa5" exitCode=0
Oct 30 22:00:26.627282 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.627153    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk" event={"ID":"9da80600-a902-41f2-847e-dbe0349d0c3c","Type":"ContainerDied","Data":"7b321108a6d5d29e49da54ee7f61fc92034989e8777b182e433b4e01193effa5"}
Oct 30 22:00:26.761394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.745659    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.849857    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9da80600-a902-41f2-847e-dbe0349d0c3c-kube-api-access-p8rww\" (UniqueName: \"kubernetes.io/projected/9da80600-a902-41f2-847e-dbe0349d0c3c-kube-api-access-p8rww\") pod \"9da80600-a902-41f2-847e-dbe0349d0c3c\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") "
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.849953    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9da80600-a902-41f2-847e-dbe0349d0c3c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da80600-a902-41f2-847e-dbe0349d0c3c-openshift-service-ca\") pod \"9da80600-a902-41f2-847e-dbe0349d0c3c\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") "
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.850019    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls\") pod \"9da80600-a902-41f2-847e-dbe0349d0c3c\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") "
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.850064    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-metrics-certs\") pod \"9da80600-a902-41f2-847e-dbe0349d0c3c\" (UID: \"9da80600-a902-41f2-847e-dbe0349d0c3c\") "
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.850958    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9da80600-a902-41f2-847e-dbe0349d0c3c-openshift-service-ca" pod "9da80600-a902-41f2-847e-dbe0349d0c3c" (UID: "9da80600-a902-41f2-847e-dbe0349d0c3c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.851183    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da80600-a902-41f2-847e-dbe0349d0c3c-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.854950    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-metrics-certs" pod "9da80600-a902-41f2-847e-dbe0349d0c3c" (UID: "9da80600-a902-41f2-847e-dbe0349d0c3c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:26.858382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.856899    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9da80600-a902-41f2-847e-dbe0349d0c3c-kube-api-access-p8rww" pod "9da80600-a902-41f2-847e-dbe0349d0c3c" (UID: "9da80600-a902-41f2-847e-dbe0349d0c3c"). InnerVolumeSpecName "kube-api-access-p8rww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:26.881144 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.874953    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls" pod "9da80600-a902-41f2-847e-dbe0349d0c3c" (UID: "9da80600-a902-41f2-847e-dbe0349d0c3c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:26.971431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.968960    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8rww\" (UniqueName: \"kubernetes.io/projected/9da80600-a902-41f2-847e-dbe0349d0c3c-kube-api-access-p8rww\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:26.971431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.968997    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:26.971431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.969013    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da80600-a902-41f2-847e-dbe0349d0c3c-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:26.994071 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.994036    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"]
Oct 30 22:00:26.995195 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.995173    5561 cpu_manager.go:414] "containerMap: removing container" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy"
Oct 30 22:00:26.995321 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.995309    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy"
Oct 30 22:00:26.995508 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.995487    5561 cpu_manager.go:414] "containerMap: removing container" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy"
Oct 30 22:00:26.995596 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.995584    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy"
Oct 30 22:00:26.995679 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.995670    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9da80600-a902-41f2-847e-dbe0349d0c3c" containerName="maas-api"
Oct 30 22:00:26.995764 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.995753    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9da80600-a902-41f2-847e-dbe0349d0c3c" containerName="maas-api"
Oct 30 22:00:26.996025 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.996014    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy"
Oct 30 22:00:26.996331 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.996316    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9da80600-a902-41f2-847e-dbe0349d0c3c" containerName="maas-api"
Oct 30 22:00:26.996468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:26.996456    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="0be698b7-5a83-4d30-8ace-917f0ac6fe1a" containerName="istio-proxy"
Oct 30 22:00:27.000424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.000399    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.055295 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.055258    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"]
Oct 30 22:00:27.058730 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.058699    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2b8311b7-dockercfg-jvs9l\""
Oct 30 22:00:27.058998 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.058939    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2b8311b7\""
Oct 30 22:00:27.070499 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.070469    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abc18d2e-6761-4564-8830-055e26806395-plugins-config-volume\") pod \"payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") " pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.070605 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.070589    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzcxx\" (UniqueName: \"kubernetes.io/projected/abc18d2e-6761-4564-8830-055e26806395-kube-api-access-gzcxx\") pod \"payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") " pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.099524 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.099489    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"]
Oct 30 22:00:27.162376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.119573    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.185375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.184057    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abc18d2e-6761-4564-8830-055e26806395-plugins-config-volume\") pod \"payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") " pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.185375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.184221    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzcxx\" (UniqueName: \"kubernetes.io/projected/abc18d2e-6761-4564-8830-055e26806395-kube-api-access-gzcxx\") pod \"payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") " pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.192478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.188001    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abc18d2e-6761-4564-8830-055e26806395-plugins-config-volume\") pod \"payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") " pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.327494 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.326972    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8031899-312a-45c2-9d82-814ffa30e5c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.327494 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.327057    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9nb\" (UniqueName: \"kubernetes.io/projected/a8031899-312a-45c2-9d82-814ffa30e5c0-kube-api-access-fn9nb\") pod \"payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.395199 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.395148    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzcxx\" (UniqueName: \"kubernetes.io/projected/abc18d2e-6761-4564-8830-055e26806395-kube-api-access-gzcxx\") pod \"payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") " pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.435947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.430086    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8031899-312a-45c2-9d82-814ffa30e5c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.445595 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.445560    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn9nb\" (UniqueName: \"kubernetes.io/projected/a8031899-312a-45c2-9d82-814ffa30e5c0-kube-api-access-fn9nb\") pod \"payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.456584 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.453252    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"]
Oct 30 22:00:27.461785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.444341    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8031899-312a-45c2-9d82-814ffa30e5c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.504979 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.504933    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn9nb\" (UniqueName: \"kubernetes.io/projected/a8031899-312a-45c2-9d82-814ffa30e5c0-kube-api-access-fn9nb\") pod \"payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.629834 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.629799    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:27.706394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.699652    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:27.708661 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.708630    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk" event={"ID":"9da80600-a902-41f2-847e-dbe0349d0c3c","Type":"ContainerDied","Data":"f895b3ceffe47297bdd0ed3a3fa7d9b9c66289ab9947eba633ba07fc2d3460e4"}
Oct 30 22:00:27.708863 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.708846    5561 scope.go:122] "RemoveContainer" containerID="7b321108a6d5d29e49da54ee7f61fc92034989e8777b182e433b4e01193effa5"
Oct 30 22:00:27.709138 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:27.709119    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"
Oct 30 22:00:28.016463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.016430    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"]
Oct 30 22:00:28.019563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.019530    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e585c53f-789474fb69-qmpjk"]
Oct 30 22:00:28.229013 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:00:28.228982    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc18d2e_6761_4564_8830_055e26806395.slice/crio-c6af05cdfb3f962222df78f410ec3b68dd1b23b61ff145c7bdd1c2aa915e05f2 WatchSource:0}: Error finding container c6af05cdfb3f962222df78f410ec3b68dd1b23b61ff145c7bdd1c2aa915e05f2: Status 404 returned error can't find the container with id c6af05cdfb3f962222df78f410ec3b68dd1b23b61ff145c7bdd1c2aa915e05f2
Oct 30 22:00:28.229126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.229026    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"]
Oct 30 22:00:28.408571 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:00:28.408529    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8031899_312a_45c2_9d82_814ffa30e5c0.slice/crio-53a58b2725583a0c5934b0b49de792812735539de3be7e4e89bfb40025e61f05 WatchSource:0}: Error finding container 53a58b2725583a0c5934b0b49de792812735539de3be7e4e89bfb40025e61f05: Status 404 returned error can't find the container with id 53a58b2725583a0c5934b0b49de792812735539de3be7e4e89bfb40025e61f05
Oct 30 22:00:28.411639 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.411609    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"]
Oct 30 22:00:28.718562 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.718518    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl" event={"ID":"abc18d2e-6761-4564-8830-055e26806395","Type":"ContainerStarted","Data":"339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db"}
Oct 30 22:00:28.719041 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.718583    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl" event={"ID":"abc18d2e-6761-4564-8830-055e26806395","Type":"ContainerStarted","Data":"c6af05cdfb3f962222df78f410ec3b68dd1b23b61ff145c7bdd1c2aa915e05f2"}
Oct 30 22:00:28.719041 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.718606    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:28.720198 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.720171    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd" event={"ID":"a8031899-312a-45c2-9d82-814ffa30e5c0","Type":"ContainerStarted","Data":"60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85"}
Oct 30 22:00:28.720322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.720215    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd" event={"ID":"a8031899-312a-45c2-9d82-814ffa30e5c0","Type":"ContainerStarted","Data":"53a58b2725583a0c5934b0b49de792812735539de3be7e4e89bfb40025e61f05"}
Oct 30 22:00:28.720322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.720235    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:28.887371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.876010    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl" podStartSLOduration=2.875986997 podStartE2EDuration="2.875986997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:28.858897192 +0000 UTC m=+1577.136590662" watchObservedRunningTime="2026-10-30 22:00:28.875986997 +0000 UTC m=+1577.153680470"
Oct 30 22:00:28.985187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:28.985127    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd" podStartSLOduration=1.985113984 podStartE2EDuration="1.985113984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:28.980823773 +0000 UTC m=+1577.258517241" watchObservedRunningTime="2026-10-30 22:00:28.985113984 +0000 UTC m=+1577.262807455"
Oct 30 22:00:29.893733 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:29.893615    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9da80600-a902-41f2-847e-dbe0349d0c3c" path="/var/lib/kubelet/pods/9da80600-a902-41f2-847e-dbe0349d0c3c/volumes"
Oct 30 22:00:30.027376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.024032    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"]
Oct 30 22:00:30.029865 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.029835    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.062771 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.062721    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7tc5\" (UniqueName: \"kubernetes.io/projected/c55a47ba-c4fd-43e6-adfe-11e33d829cea-kube-api-access-k7tc5\") pod \"payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.063245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.063223    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55a47ba-c4fd-43e6-adfe-11e33d829cea-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.083376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.080056    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"]
Oct 30 22:00:30.097372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.085711    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-87e2c278-dockercfg-ctc9t\""
Oct 30 22:00:30.153441 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.145463    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-87e2c278\""
Oct 30 22:00:30.184401 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.165232    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tc5\" (UniqueName: \"kubernetes.io/projected/c55a47ba-c4fd-43e6-adfe-11e33d829cea-kube-api-access-k7tc5\") pod \"payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.184401 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.165494    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55a47ba-c4fd-43e6-adfe-11e33d829cea-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.198365 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.192264    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55a47ba-c4fd-43e6-adfe-11e33d829cea-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.198365 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.193940    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"]
Oct 30 22:00:30.280446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.268545    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7tc5\" (UniqueName: \"kubernetes.io/projected/c55a47ba-c4fd-43e6-adfe-11e33d829cea-kube-api-access-k7tc5\") pod \"payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.301409 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.297288    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:30.306384 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.305549    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"]
Oct 30 22:00:30.397291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.386538    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5d7920-c867-4deb-b358-b364ab788f64-plugins-config-volume\") pod \"payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") " pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:30.423129 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.422813    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:30.508399 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.508330    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dshsz\" (UniqueName: \"kubernetes.io/projected/de5d7920-c867-4deb-b358-b364ab788f64-kube-api-access-dshsz\") pod \"payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") " pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:30.508612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.508585    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5d7920-c867-4deb-b358-b364ab788f64-plugins-config-volume\") pod \"payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") " pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:30.509914 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.509886    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5d7920-c867-4deb-b358-b364ab788f64-plugins-config-volume\") pod \"payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") " pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:30.620252 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.618799    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dshsz\" (UniqueName: \"kubernetes.io/projected/de5d7920-c867-4deb-b358-b364ab788f64-kube-api-access-dshsz\") pod \"payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") " pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:30.682329 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.682256    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dshsz\" (UniqueName: \"kubernetes.io/projected/de5d7920-c867-4deb-b358-b364ab788f64-kube-api-access-dshsz\") pod \"payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") " pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:30.716945 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:30.716908    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:31.033497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:31.033329    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"]
Oct 30 22:00:31.148856 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:31.148802    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"]
Oct 30 22:00:31.203618 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:00:31.203587    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5d7920_c867_4deb_b358_b364ab788f64.slice/crio-d7432c070e8c214a373388db1fa1ef29ad1e674e0eb8470b09cdd193407441ea WatchSource:0}: Error finding container d7432c070e8c214a373388db1fa1ef29ad1e674e0eb8470b09cdd193407441ea: Status 404 returned error can't find the container with id d7432c070e8c214a373388db1fa1ef29ad1e674e0eb8470b09cdd193407441ea
Oct 30 22:00:32.102017 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:32.101978    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd" event={"ID":"c55a47ba-c4fd-43e6-adfe-11e33d829cea","Type":"ContainerStarted","Data":"fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01"}
Oct 30 22:00:32.102683 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:32.102040    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd" event={"ID":"c55a47ba-c4fd-43e6-adfe-11e33d829cea","Type":"ContainerStarted","Data":"b7800ac5053c57bd25cb48bb4aca7101ff61bf8b5a605883e09d052d8cd5622f"}
Oct 30 22:00:32.102683 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:32.102072    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:32.104176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:32.104147    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m" event={"ID":"de5d7920-c867-4deb-b358-b364ab788f64","Type":"ContainerStarted","Data":"4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3"}
Oct 30 22:00:32.104328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:32.104189    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m" event={"ID":"de5d7920-c867-4deb-b358-b364ab788f64","Type":"ContainerStarted","Data":"d7432c070e8c214a373388db1fa1ef29ad1e674e0eb8470b09cdd193407441ea"}
Oct 30 22:00:32.104451 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:32.104432    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:32.231812 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:32.231757    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd" podStartSLOduration=3.231743607 podStartE2EDuration="3.231743607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:32.229151794 +0000 UTC m=+1580.506845265" watchObservedRunningTime="2026-10-30 22:00:32.231743607 +0000 UTC m=+1580.509437078"
Oct 30 22:00:39.856493 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:39.856464    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:00:39.856866 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:39.856525    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:00:39.972881 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:39.972831    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m" podStartSLOduration=9.972818764 podStartE2EDuration="9.972818764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:32.372731177 +0000 UTC m=+1580.650424649" watchObservedRunningTime="2026-10-30 22:00:39.972818764 +0000 UTC m=+1588.250512235"
Oct 30 22:00:42.823953 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:42.823921    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:00:43.115675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:43.115601    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:00:43.115675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:00:43.115673    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:00:52.983536 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:00:52.983497    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:01:04.125503 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.125462    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"]
Oct 30 22:01:04.130315 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.130289    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.183872 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.183838    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2b8311b7-serving-cert\""
Oct 30 22:01:04.233279 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.233242    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.233575 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.233555    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4qh\" (UniqueName: \"kubernetes.io/projected/9cdd3026-6800-40ab-989f-7c336fe9bf86-kube-api-access-nz4qh\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.233644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.233594    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdd3026-6800-40ab-989f-7c336fe9bf86-openshift-service-ca\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.233644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.233637    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-metrics-certs\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.238067 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.238042    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"]
Oct 30 22:01:04.335699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.335665    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.335877 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:01:04.335813    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-serving-cert: secret "maas-api-e2e-derive-2b8311b7-serving-cert" not found
Oct 30 22:01:04.335877 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.335848    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4qh\" (UniqueName: \"kubernetes.io/projected/9cdd3026-6800-40ab-989f-7c336fe9bf86-kube-api-access-nz4qh\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.335966 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:01:04.335885    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls podName:9cdd3026-6800-40ab-989f-7c336fe9bf86 nodeName:}" failed. No retries permitted until 2026-10-30 22:01:04.835868546 +0000 UTC m=+1613.113561998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls") pod "maas-api-e2e-derive-2b8311b7-765667c68-7vrv4" (UID: "9cdd3026-6800-40ab-989f-7c336fe9bf86") : secret "maas-api-e2e-derive-2b8311b7-serving-cert" not found
Oct 30 22:01:04.335966 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.335904    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdd3026-6800-40ab-989f-7c336fe9bf86-openshift-service-ca\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.335966 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.335939    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-metrics-certs\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.336705 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.336673    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdd3026-6800-40ab-989f-7c336fe9bf86-openshift-service-ca\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.338492 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.338469    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-metrics-certs\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.448982 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.442542    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4qh\" (UniqueName: \"kubernetes.io/projected/9cdd3026-6800-40ab-989f-7c336fe9bf86-kube-api-access-nz4qh\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.845651 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.845569    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:04.848142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:04.848113    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls\") pod \"maas-api-e2e-derive-2b8311b7-765667c68-7vrv4\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:05.044298 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:05.044263    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:05.428396 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:05.428367    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"]
Oct 30 22:01:05.428825 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:01:05.428793    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cdd3026_6800_40ab_989f_7c336fe9bf86.slice/crio-9e88f0f676e4b1046e29f095ee1110ccc2db12627337373062ca10b1173d28c9 WatchSource:0}: Error finding container 9e88f0f676e4b1046e29f095ee1110ccc2db12627337373062ca10b1173d28c9: Status 404 returned error can't find the container with id 9e88f0f676e4b1046e29f095ee1110ccc2db12627337373062ca10b1173d28c9
Oct 30 22:01:06.375756 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:06.375711    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4" event={"ID":"9cdd3026-6800-40ab-989f-7c336fe9bf86","Type":"ContainerStarted","Data":"9e88f0f676e4b1046e29f095ee1110ccc2db12627337373062ca10b1173d28c9"}
Oct 30 22:01:07.388849 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:07.388802    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4" event={"ID":"9cdd3026-6800-40ab-989f-7c336fe9bf86","Type":"ContainerStarted","Data":"56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6"}
Oct 30 22:01:07.389328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:07.389117    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:07.508963 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:07.508907    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4" podStartSLOduration=2.657725907 podStartE2EDuration="3.508893707s" totalImagesPullingTime="851.1678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:07.505787052 +0000 UTC m=+1615.783480523" watchObservedRunningTime="2026-10-30 22:01:07.508893707 +0000 UTC m=+1615.786587195"
Oct 30 22:01:13.401876 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:13.401843    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:01:13.905848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:13.905814    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"]
Oct 30 22:01:13.911614 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:13.911589    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:13.965246 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:13.965213    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-87e2c278-serving-cert\""
Oct 30 22:01:13.967215 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:13.967190    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"]
Oct 30 22:01:14.055326 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.055291    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.055656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.055633    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-metrics-certs\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.055800 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.055780    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnl4g\" (UniqueName: \"kubernetes.io/projected/480dee01-cb09-401a-8ca5-7daeee0251dd-kube-api-access-bnl4g\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.055904 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.055890    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480dee01-cb09-401a-8ca5-7daeee0251dd-openshift-service-ca\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.168521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.167234    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-metrics-certs\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.170225 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.170192    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnl4g\" (UniqueName: \"kubernetes.io/projected/480dee01-cb09-401a-8ca5-7daeee0251dd-kube-api-access-bnl4g\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.194336 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.194301    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480dee01-cb09-401a-8ca5-7daeee0251dd-openshift-service-ca\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.201198 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.201145    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.201625 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.200667    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-metrics-certs\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.201781 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.195521    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480dee01-cb09-401a-8ca5-7daeee0251dd-openshift-service-ca\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.202019 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:01:14.201989    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-serving-cert: secret "maas-api-e2e-migrate-87e2c278-serving-cert" not found
Oct 30 22:01:14.202277 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:01:14.202260    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls podName:480dee01-cb09-401a-8ca5-7daeee0251dd nodeName:}" failed. No retries permitted until 2026-10-30 22:01:14.702237495 +0000 UTC m=+1622.979930943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls") pod "maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll" (UID: "480dee01-cb09-401a-8ca5-7daeee0251dd") : secret "maas-api-e2e-migrate-87e2c278-serving-cert" not found
Oct 30 22:01:14.242542 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.242506    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnl4g\" (UniqueName: \"kubernetes.io/projected/480dee01-cb09-401a-8ca5-7daeee0251dd-kube-api-access-bnl4g\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.718112 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.718063    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.720603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.720571    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls\") pod \"maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:14.825922 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:14.825876    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:15.202847 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:01:15.202820    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480dee01_cb09_401a_8ca5_7daeee0251dd.slice/crio-d61ed9c0e2aab0677950ec6f58851cc212ae002d72bf856bb124dffaf9be1cb6 WatchSource:0}: Error finding container d61ed9c0e2aab0677950ec6f58851cc212ae002d72bf856bb124dffaf9be1cb6: Status 404 returned error can't find the container with id d61ed9c0e2aab0677950ec6f58851cc212ae002d72bf856bb124dffaf9be1cb6
Oct 30 22:01:15.203002 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:15.202914    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"]
Oct 30 22:01:15.524514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:15.524432    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll" event={"ID":"480dee01-cb09-401a-8ca5-7daeee0251dd","Type":"ContainerStarted","Data":"d61ed9c0e2aab0677950ec6f58851cc212ae002d72bf856bb124dffaf9be1cb6"}
Oct 30 22:01:16.532753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:16.532711    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll" event={"ID":"480dee01-cb09-401a-8ca5-7daeee0251dd","Type":"ContainerStarted","Data":"8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88"}
Oct 30 22:01:16.533168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:16.532806    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:16.656741 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:16.656685    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll" podStartSLOduration=2.754307805 podStartE2EDuration="3.656671656s" totalImagesPullingTime="902.363851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:16.652579911 +0000 UTC m=+1624.930273382" watchObservedRunningTime="2026-10-30 22:01:16.656671656 +0000 UTC m=+1624.934365126"
Oct 30 22:01:22.545389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:22.545289    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:01:22.982030 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:01:22.981989    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:01:28.664373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:28.662692    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:01:31.910376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:31.909588    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 22:01:32.101412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.089207    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 22:01:32.110867 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.110820    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.133661 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.133608    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.173517 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.161944    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.224374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.223705    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.224374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.224017    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.238478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.238428    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.285510 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.285448    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.286118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.286090    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:01:32.590435 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.589465    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.930052 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.916442    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.930052 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.916933    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 22:01:32.954211 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.934267    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:32.954211 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:32.942306    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:33.128735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.119875    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:33.128735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.120222    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 22:01:33.178381 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.177883    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 22:01:33.178381 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.178294    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 22:01:33.281760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.245422    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 30 22:01:33.281760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.245858    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 22:01:33.281760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.279111    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:33.281760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.279508    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 22:01:33.290372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.287508    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:33.290372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.287865    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 22:01:33.460479 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.412338    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 22:01:33.669243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.658439    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:33.669243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.659217    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 22:01:33.669243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.659305    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 22:01:33.832374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.831561    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 22:01:33.922459 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.919954    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:33.938005 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.937957    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:33.941524 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:33.941498    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 22:01:34.287302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:34.287185    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:34.561670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:34.386182    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:34.561670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:34.392422    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:01:34.771369 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:34.771116    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 22:01:35.023429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:34.866814    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 22:01:35.023429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:34.996819    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 22:01:35.023429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:34.998289    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 22:01:35.172698 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.111538    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 22:01:35.172698 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.111916    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:35.404085 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.404016    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:35.453417 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.436414    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:35.508385 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.500040    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:35.508385 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.500331    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Oct 30 22:01:35.508385 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.506507    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 22:01:35.773427 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.748087    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:35.839889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.773699    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:35.839889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.822575    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:35.915376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:35.879388    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 22:01:36.149641 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.025892    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 22:01:36.149641 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.067471    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:36.231380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.230488    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 22:01:36.437422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.436946    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:36.535374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.532559    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:36.535374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.533059    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:36.706426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.704330    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"]
Oct 30 22:01:36.712223 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.712200    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"]
Oct 30 22:01:36.712497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.712483    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 22:01:36.712657 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.712643    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 22:01:36.712793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.712780    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 22:01:36.712923 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.712910    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 22:01:36.713043 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.713032    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 22:01:36.713171 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.713160    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 22:01:36.717121 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.714547    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.723570 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.723071    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 22:01:36.734890 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.734500    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6"
Oct 30 22:01:36.743381 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.735563    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 22:01:36.746006 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.745984    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 22:01:36.772374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.770033    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-326d76-openshift-default-dockercfg-7lwbj\""
Oct 30 22:01:36.799168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.799138    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-certs\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.799515 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.799495    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-envoy\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.800440 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.800419    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-socket\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.800687 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.800668    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bznjm\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-kube-api-access-bznjm\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.800955 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.800939    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-token\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.801116 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.801101    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-credential-socket\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.801224 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.801211    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef9544f-6a1c-4289-997c-532d42ba7d38-istiod-ca-cert\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.801466 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.801452    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-data\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.801601 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.801588    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-podinfo\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.903840 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.903801    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-token\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.904177 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.904152    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-credential-socket\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.904328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.904309    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef9544f-6a1c-4289-997c-532d42ba7d38-istiod-ca-cert\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.904587 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.904559    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-data\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.904736 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.904719    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-credential-socket\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.904857 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.904734    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-podinfo\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905085    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-certs\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905242 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905226    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-envoy\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905368 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905341    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-socket\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905488    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bznjm\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-kube-api-access-bznjm\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905637 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905611    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-certs\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905373    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-data\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905114    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef9544f-6a1c-4289-997c-532d42ba7d38-istiod-ca-cert\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.905940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.905920    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-socket\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.907819 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.907793    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-envoy\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:36.908333 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:36.908307    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-podinfo\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:37.028487 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:37.027459    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bznjm\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-kube-api-access-bznjm\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:37.028859 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:37.028829    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-token\") pod \"e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") " pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:37.068708 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:37.068668    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:37.721343 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:37.721302    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"]
Oct 30 22:01:37.729988 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:01:37.727910    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef9544f_6a1c_4289_997c_532d42ba7d38.slice/crio-691452a493ec881320c8580942eeebda9dee5c02524c785bfab25cf25137806f WatchSource:0}: Error finding container 691452a493ec881320c8580942eeebda9dee5c02524c785bfab25cf25137806f: Status 404 returned error can't find the container with id 691452a493ec881320c8580942eeebda9dee5c02524c785bfab25cf25137806f
Oct 30 22:01:37.732331 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:37.732286    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:37.732481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:37.732403    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:37.732481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:37.732472    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:38.743946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:38.743852    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" event={"ID":"fef9544f-6a1c-4289-997c-532d42ba7d38","Type":"ContainerStarted","Data":"cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d"}
Oct 30 22:01:38.744553 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:38.744524    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" event={"ID":"fef9544f-6a1c-4289-997c-532d42ba7d38","Type":"ContainerStarted","Data":"691452a493ec881320c8580942eeebda9dee5c02524c785bfab25cf25137806f"}
Oct 30 22:01:38.869698 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:38.869624    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podStartSLOduration=3.869605383 podStartE2EDuration="3.869605383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:38.867605032 +0000 UTC m=+1647.145298528" watchObservedRunningTime="2026-10-30 22:01:38.869605383 +0000 UTC m=+1647.147298857"
Oct 30 22:01:39.072340 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:39.070640    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:39.089981 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:39.088790    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:39.755418 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:39.755386    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:39.756959 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:39.756925    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:01:40.651546 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:40.651502    5561 patch_prober.go:29] interesting pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:01:40.651732 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:40.651557    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:41.711962 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:41.711022    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"]
Oct 30 22:01:42.045588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.044993    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.096414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.074439    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78de8a1e-3c7c-4a67-9228-355e9083e0d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.097067 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.097027    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzqh4\" (UniqueName: \"kubernetes.io/projected/78de8a1e-3c7c-4a67-9228-355e9083e0d5-kube-api-access-fzqh4\") pod \"payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.178599 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.165732    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e2deec91\""
Oct 30 22:01:42.178599 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.165858    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e2deec91-dockercfg-pmk8t\""
Oct 30 22:01:42.204615 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.202993    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzqh4\" (UniqueName: \"kubernetes.io/projected/78de8a1e-3c7c-4a67-9228-355e9083e0d5-kube-api-access-fzqh4\") pod \"payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.204615 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.203260    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78de8a1e-3c7c-4a67-9228-355e9083e0d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.206103 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.205845    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78de8a1e-3c7c-4a67-9228-355e9083e0d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.211796 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.211773    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"]
Oct 30 22:01:42.212099 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.212083    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"]
Oct 30 22:01:42.225941 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.224896    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.226145 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.224799    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"]
Oct 30 22:01:42.264703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.264669    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzqh4\" (UniqueName: \"kubernetes.io/projected/78de8a1e-3c7c-4a67-9228-355e9083e0d5-kube-api-access-fzqh4\") pod \"payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.307071 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.306992    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99bcb54-f104-4351-b3ca-8b132b8f662d-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.307385 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.307361    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48g9b\" (UniqueName: \"kubernetes.io/projected/d99bcb54-f104-4351-b3ca-8b132b8f662d-kube-api-access-48g9b\") pod \"payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.408977 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.408935    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48g9b\" (UniqueName: \"kubernetes.io/projected/d99bcb54-f104-4351-b3ca-8b132b8f662d-kube-api-access-48g9b\") pod \"payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.409175 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.409123    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99bcb54-f104-4351-b3ca-8b132b8f662d-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.409926 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.409903    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99bcb54-f104-4351-b3ca-8b132b8f662d-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.467756 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.467711    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48g9b\" (UniqueName: \"kubernetes.io/projected/d99bcb54-f104-4351-b3ca-8b132b8f662d-kube-api-access-48g9b\") pod \"payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.478482 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.478451    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:42.558090 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.554830    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:42.868642 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.868612    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"]
Oct 30 22:01:42.894400 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:01:42.894333    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78de8a1e_3c7c_4a67_9228_355e9083e0d5.slice/crio-a563fceafa959eb9df0622d06457ebbfd7376f6054713891bb2a73af15da231d WatchSource:0}: Error finding container a563fceafa959eb9df0622d06457ebbfd7376f6054713891bb2a73af15da231d: Status 404 returned error can't find the container with id a563fceafa959eb9df0622d06457ebbfd7376f6054713891bb2a73af15da231d
Oct 30 22:01:42.964780 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.964748    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2" event={"ID":"78de8a1e-3c7c-4a67-9228-355e9083e0d5","Type":"ContainerStarted","Data":"a563fceafa959eb9df0622d06457ebbfd7376f6054713891bb2a73af15da231d"}
Oct 30 22:01:42.966898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.966871    5561 generic.go:356] "Generic (PLEG): container finished" podID="cfb83f97-7336-40d1-a749-511d9118b63d" containerID="f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3" exitCode=0
Oct 30 22:01:42.967008 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.966939    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" event={"ID":"cfb83f97-7336-40d1-a749-511d9118b63d","Type":"ContainerDied","Data":"f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3"}
Oct 30 22:01:42.967411 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.967397    5561 scope.go:122] "RemoveContainer" containerID="f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3"
Oct 30 22:01:42.969065 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.969037    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:42.969137 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.969104    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:42.969173 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.969135    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:42.988009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:42.987975    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"]
Oct 30 22:01:42.989776 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:01:42.989755    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99bcb54_f104_4351_b3ca_8b132b8f662d.slice/crio-734f90c7990e9a032578514ac9fd97fb085428cd37df9810aed77e5df8091071 WatchSource:0}: Error finding container 734f90c7990e9a032578514ac9fd97fb085428cd37df9810aed77e5df8091071: Status 404 returned error can't find the container with id 734f90c7990e9a032578514ac9fd97fb085428cd37df9810aed77e5df8091071
Oct 30 22:01:43.980058 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.980012    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh" event={"ID":"d99bcb54-f104-4351-b3ca-8b132b8f662d","Type":"ContainerStarted","Data":"63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85"}
Oct 30 22:01:43.980666 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.980082    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh" event={"ID":"d99bcb54-f104-4351-b3ca-8b132b8f662d","Type":"ContainerStarted","Data":"734f90c7990e9a032578514ac9fd97fb085428cd37df9810aed77e5df8091071"}
Oct 30 22:01:43.980666 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.980205    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:43.981761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.981739    5561 generic.go:356] "Generic (PLEG): container finished" podID="70638018-48d1-4ef3-b021-b974ee34d87a" containerID="77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e" exitCode=0
Oct 30 22:01:43.981865 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.981818    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" event={"ID":"70638018-48d1-4ef3-b021-b974ee34d87a","Type":"ContainerDied","Data":"77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e"}
Oct 30 22:01:43.982186 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.982169    5561 scope.go:122] "RemoveContainer" containerID="77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e"
Oct 30 22:01:43.983729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.983700    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:43.983829 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.983777    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:43.983829 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.983817    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:43.984292 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.984272    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" event={"ID":"cfb83f97-7336-40d1-a749-511d9118b63d","Type":"ContainerStarted","Data":"9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3"}
Oct 30 22:01:43.986167 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.986142    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2" event={"ID":"78de8a1e-3c7c-4a67-9228-355e9083e0d5","Type":"ContainerStarted","Data":"18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d"}
Oct 30 22:01:43.986310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:43.986259    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:01:44.100264 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:44.100218    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh" podStartSLOduration=3.100202357 podStartE2EDuration="3.100202357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:44.098265995 +0000 UTC m=+1652.375959471" watchObservedRunningTime="2026-10-30 22:01:44.100202357 +0000 UTC m=+1652.377895828"
Oct 30 22:01:44.588103 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:44.588043    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2" podStartSLOduration=3.588027918 podStartE2EDuration="3.588027918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:44.586323406 +0000 UTC m=+1652.864016879" watchObservedRunningTime="2026-10-30 22:01:44.588027918 +0000 UTC m=+1652.865721389"
Oct 30 22:01:44.643224 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:44.643195    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:01:44.645486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:44.643881    5561 patch_prober.go:29] interesting pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:01:44.645486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:44.643934    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:01:44.996950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:44.996899    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" event={"ID":"70638018-48d1-4ef3-b021-b974ee34d87a","Type":"ContainerStarted","Data":"e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c"}
Oct 30 22:01:45.164378 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:45.158689    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:01:45.165509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:45.165473    5561 patch_prober.go:29] interesting pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:01:45.165736 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:45.165704    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:01:45.653507 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:45.653463    5561 patch_prober.go:29] interesting pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:01:45.653816 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:45.653786    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:01:46.168375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:46.165739    5561 patch_prober.go:29] interesting pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:01:46.168375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:46.165801    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:01:46.721267 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:46.721229    5561 patch_prober.go:29] interesting pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:01:46.721683 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:46.721632    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:01:47.102431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.087433    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"]
Oct 30 22:01:47.190376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.169461    5561 patch_prober.go:29] interesting pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:01:47.190376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.169522    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:01:47.256794 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.256759    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:47.273387 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.269228    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9780746-8d6a-4e0f-a91d-f14f18d68629-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:47.273387 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.269866    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zlzx\" (UniqueName: \"kubernetes.io/projected/c9780746-8d6a-4e0f-a91d-f14f18d68629-kube-api-access-6zlzx\") pod \"payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:47.316432 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.307235    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"]
Oct 30 22:01:47.348374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.337510    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-326d76-dockercfg-6z5c7\""
Oct 30 22:01:47.434093 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.433176    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9780746-8d6a-4e0f-a91d-f14f18d68629-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:47.475372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.470623    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"]
Oct 30 22:01:47.477628 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.477587    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zlzx\" (UniqueName: \"kubernetes.io/projected/c9780746-8d6a-4e0f-a91d-f14f18d68629-kube-api-access-6zlzx\") pod \"payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:47.557423 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.538253    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-326d76\""
Oct 30 22:01:47.630373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.617164    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:47.686429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.637987    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9780746-8d6a-4e0f-a91d-f14f18d68629-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:47.710597 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.710541    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:47.715772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.714539    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 22:01:47.768373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.767775    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6" podUID="1301ca09-349b-414d-bdf2-8297fa20c2cd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 22:01:47.768373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.768111    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 22:01:47.814373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.812147    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:47.814373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.814096    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zlzx\" (UniqueName: \"kubernetes.io/projected/c9780746-8d6a-4e0f-a91d-f14f18d68629-kube-api-access-6zlzx\") pod \"payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:47.815775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.814695    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:47.822909 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.817088    5561 patch_prober.go:29] interesting pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:01:47.822909 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.822468    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:01:47.822909 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.819212    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"]
Oct 30 22:01:47.910143 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.909417    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q982r\" (UniqueName: \"kubernetes.io/projected/4db227fc-374f-45bd-aa45-4c5abbe45e74-kube-api-access-q982r\") pod \"payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:47.911216 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.910961    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4db227fc-374f-45bd-aa45-4c5abbe45e74-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:47.989446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:47.970525    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:48.013636 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.013600    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q982r\" (UniqueName: \"kubernetes.io/projected/4db227fc-374f-45bd-aa45-4c5abbe45e74-kube-api-access-q982r\") pod \"payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:48.013771 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.013676    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4db227fc-374f-45bd-aa45-4c5abbe45e74-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:48.014397 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.014339    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4db227fc-374f-45bd-aa45-4c5abbe45e74-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:48.085375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.085066    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q982r\" (UniqueName: \"kubernetes.io/projected/4db227fc-374f-45bd-aa45-4c5abbe45e74-kube-api-access-q982r\") pod \"payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:48.171380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.170758    5561 patch_prober.go:29] interesting pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:01:48.171380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.170833    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:01:48.205162 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.205121    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:48.441115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.441088    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"]
Oct 30 22:01:48.442012 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:01:48.441980    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9780746_8d6a_4e0f_a91d_f14f18d68629.slice/crio-5cc2b1f476fa3ebcc68e14a23494a3fd6b0cbf305888b12a95fad2b17c8471b6 WatchSource:0}: Error finding container 5cc2b1f476fa3ebcc68e14a23494a3fd6b0cbf305888b12a95fad2b17c8471b6: Status 404 returned error can't find the container with id 5cc2b1f476fa3ebcc68e14a23494a3fd6b0cbf305888b12a95fad2b17c8471b6
Oct 30 22:01:48.642646 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:01:48.642617    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db227fc_374f_45bd_aa45_4c5abbe45e74.slice/crio-467f1cacb7ac16abc48094641e4a2f43a4d22442060a423ddc4a62a24545f1cd WatchSource:0}: Error finding container 467f1cacb7ac16abc48094641e4a2f43a4d22442060a423ddc4a62a24545f1cd: Status 404 returned error can't find the container with id 467f1cacb7ac16abc48094641e4a2f43a4d22442060a423ddc4a62a24545f1cd
Oct 30 22:01:48.642861 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.642830    5561 patch_prober.go:29] interesting pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:01:48.642952 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.642874    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:01:48.653393 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:48.643470    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"]
Oct 30 22:01:49.164065 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.164027    5561 patch_prober.go:29] interesting pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:01:49.164762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.164342    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:01:49.211729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.211698    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p" event={"ID":"c9780746-8d6a-4e0f-a91d-f14f18d68629","Type":"ContainerStarted","Data":"e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7"}
Oct 30 22:01:49.212166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.212139    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p" event={"ID":"c9780746-8d6a-4e0f-a91d-f14f18d68629","Type":"ContainerStarted","Data":"5cc2b1f476fa3ebcc68e14a23494a3fd6b0cbf305888b12a95fad2b17c8471b6"}
Oct 30 22:01:49.212264 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.212185    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:01:49.213482 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.213454    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp" event={"ID":"4db227fc-374f-45bd-aa45-4c5abbe45e74","Type":"ContainerStarted","Data":"cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af"}
Oct 30 22:01:49.213616 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.213506    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:01:49.213616 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.213532    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp" event={"ID":"4db227fc-374f-45bd-aa45-4c5abbe45e74","Type":"ContainerStarted","Data":"467f1cacb7ac16abc48094641e4a2f43a4d22442060a423ddc4a62a24545f1cd"}
Oct 30 22:01:49.339935 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.339885    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p" podStartSLOduration=3.339870807 podStartE2EDuration="3.339870807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:49.336780346 +0000 UTC m=+1657.614473817" watchObservedRunningTime="2026-10-30 22:01:49.339870807 +0000 UTC m=+1657.617564277"
Oct 30 22:01:49.483514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.483394    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp" podStartSLOduration=2.483374901 podStartE2EDuration="2.483374901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:49.465935519 +0000 UTC m=+1657.743628992" watchObservedRunningTime="2026-10-30 22:01:49.483374901 +0000 UTC m=+1657.761068372"
Oct 30 22:01:49.670524 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.670489    5561 patch_prober.go:29] interesting pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:01:49.677892 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:49.677852    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:01:50.163376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:50.160937    5561 patch_prober.go:29] interesting pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:01:50.163376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:50.161023    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:01:50.668751 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:50.668715    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:01:50.669780 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:50.669756    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:01:50.670740 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:50.670721    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:01:51.159405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:51.157743    5561 patch_prober.go:29] interesting pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:01:51.159405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:51.157816    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:01:51.585335 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:51.585259    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:01:52.159477 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:52.159450    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:01:52.299374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:52.289414    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:01:52.983363 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:01:52.983312    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:01:54.999928 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:54.999897    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:01:55.000314 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:01:54.999974    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:02:00.224409 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:00.224376    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:02:00.224876 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:00.224455    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:02:15.723938 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:15.723793    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"]
Oct 30 22:02:15.857918 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:15.857884    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"]
Oct 30 22:02:18.526471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:18.526434    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"]
Oct 30 22:02:18.526857 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:18.526641    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy" containerID="cri-o://c51f59f1aaad813a1bab35858caf2c678002c18ede6ae099bce13a9f10d8c8a2" gracePeriod=30
Oct 30 22:02:18.807904 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:18.807824    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"]
Oct 30 22:02:18.808064 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:18.808027    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" containerID="cri-o://15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da" gracePeriod=30
Oct 30 22:02:20.724144 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:20.724060    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl" podUID="abc18d2e-6761-4564-8830-055e26806395" containerName="payload-processing" containerID="cri-o://339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db" gracePeriod=25
Oct 30 22:02:20.858672 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:20.858568    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd" podUID="a8031899-312a-45c2-9d82-814ffa30e5c0" containerName="payload-pre-processing" containerID="cri-o://60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85" gracePeriod=25
Oct 30 22:02:21.127864 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.127566    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:02:21.145539 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.145506    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abc18d2e-6761-4564-8830-055e26806395-kube-api-access-gzcxx\" (UniqueName: \"kubernetes.io/projected/abc18d2e-6761-4564-8830-055e26806395-kube-api-access-gzcxx\") pod \"abc18d2e-6761-4564-8830-055e26806395\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") "
Oct 30 22:02:21.232977 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.225909    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc18d2e-6761-4564-8830-055e26806395-kube-api-access-gzcxx" pod "abc18d2e-6761-4564-8830-055e26806395" (UID: "abc18d2e-6761-4564-8830-055e26806395"). InnerVolumeSpecName "kube-api-access-gzcxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:21.270268 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.267086    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abc18d2e-6761-4564-8830-055e26806395-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abc18d2e-6761-4564-8830-055e26806395-plugins-config-volume\") pod \"abc18d2e-6761-4564-8830-055e26806395\" (UID: \"abc18d2e-6761-4564-8830-055e26806395\") "
Oct 30 22:02:21.285231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.285185    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abc18d2e-6761-4564-8830-055e26806395-plugins-config-volume" pod "abc18d2e-6761-4564-8830-055e26806395" (UID: "abc18d2e-6761-4564-8830-055e26806395"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:21.294247 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.294211    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzcxx\" (UniqueName: \"kubernetes.io/projected/abc18d2e-6761-4564-8830-055e26806395-kube-api-access-gzcxx\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:21.294692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.294666    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abc18d2e-6761-4564-8830-055e26806395-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:21.404725 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.404697    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:02:21.496986 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.496916    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8031899-312a-45c2-9d82-814ffa30e5c0-kube-api-access-fn9nb\" (UniqueName: \"kubernetes.io/projected/a8031899-312a-45c2-9d82-814ffa30e5c0-kube-api-access-fn9nb\") pod \"a8031899-312a-45c2-9d82-814ffa30e5c0\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") "
Oct 30 22:02:21.497122 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.497097    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8031899-312a-45c2-9d82-814ffa30e5c0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8031899-312a-45c2-9d82-814ffa30e5c0-plugins-config-volume\") pod \"a8031899-312a-45c2-9d82-814ffa30e5c0\" (UID: \"a8031899-312a-45c2-9d82-814ffa30e5c0\") "
Oct 30 22:02:21.497488 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.497462    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8031899-312a-45c2-9d82-814ffa30e5c0-plugins-config-volume" pod "a8031899-312a-45c2-9d82-814ffa30e5c0" (UID: "a8031899-312a-45c2-9d82-814ffa30e5c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:21.497762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.497742    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8031899-312a-45c2-9d82-814ffa30e5c0-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:21.498957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.498938    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8031899-312a-45c2-9d82-814ffa30e5c0-kube-api-access-fn9nb" pod "a8031899-312a-45c2-9d82-814ffa30e5c0" (UID: "a8031899-312a-45c2-9d82-814ffa30e5c0"). InnerVolumeSpecName "kube-api-access-fn9nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:21.599530 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.599497    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn9nb\" (UniqueName: \"kubernetes.io/projected/a8031899-312a-45c2-9d82-814ffa30e5c0-kube-api-access-fn9nb\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:21.644987 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.644954    5561 generic.go:356] "Generic (PLEG): container finished" podID="abc18d2e-6761-4564-8830-055e26806395" containerID="339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db" exitCode=0
Oct 30 22:02:21.645165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.645016    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"
Oct 30 22:02:21.645165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.645050    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl" event={"ID":"abc18d2e-6761-4564-8830-055e26806395","Type":"ContainerDied","Data":"339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db"}
Oct 30 22:02:21.645165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.645133    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl" event={"ID":"abc18d2e-6761-4564-8830-055e26806395","Type":"ContainerDied","Data":"c6af05cdfb3f962222df78f410ec3b68dd1b23b61ff145c7bdd1c2aa915e05f2"}
Oct 30 22:02:21.645165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.645157    5561 scope.go:122] "RemoveContainer" containerID="339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db"
Oct 30 22:02:21.646639 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.646618    5561 generic.go:356] "Generic (PLEG): container finished" podID="a8031899-312a-45c2-9d82-814ffa30e5c0" containerID="60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85" exitCode=0
Oct 30 22:02:21.646752 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.646648    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd" event={"ID":"a8031899-312a-45c2-9d82-814ffa30e5c0","Type":"ContainerDied","Data":"60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85"}
Oct 30 22:02:21.646752 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.646681    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"
Oct 30 22:02:21.646752 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.646698    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd" event={"ID":"a8031899-312a-45c2-9d82-814ffa30e5c0","Type":"ContainerDied","Data":"53a58b2725583a0c5934b0b49de792812735539de3be7e4e89bfb40025e61f05"}
Oct 30 22:02:21.656813 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.656794    5561 scope.go:122] "RemoveContainer" containerID="339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db"
Oct 30 22:02:21.657084 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:21.657065    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db\": container with ID starting with 339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db not found: ID does not exist" containerID="339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db"
Oct 30 22:02:21.657169 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.657091    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db"} err="failed to get container status \"339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db\": rpc error: code = NotFound desc = could not find container \"339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db\": container with ID starting with 339f766fad1389d06fa956415f814c99b009c1584eaed4a627ab941d6da5b0db not found: ID does not exist"
Oct 30 22:02:21.657169 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.657108    5561 scope.go:122] "RemoveContainer" containerID="60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85"
Oct 30 22:02:21.666584 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.666569    5561 scope.go:122] "RemoveContainer" containerID="60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85"
Oct 30 22:02:21.666855 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:21.666838    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85\": container with ID starting with 60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85 not found: ID does not exist" containerID="60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85"
Oct 30 22:02:21.666895 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.666874    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85"} err="failed to get container status \"60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85\": rpc error: code = NotFound desc = could not find container \"60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85\": container with ID starting with 60397213982af9d045fa9df3668afbcca05370e4f62a6da2f2d4e78a55c91b85 not found: ID does not exist"
Oct 30 22:02:21.812542 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.812475    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"]
Oct 30 22:02:21.815944 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.815922    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd"]
Oct 30 22:02:21.847216 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.847180    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8031899-312a-45c2-9d82-814ffa30e5c0" path="/var/lib/kubelet/pods/a8031899-312a-45c2-9d82-814ffa30e5c0/volumes"
Oct 30 22:02:21.926788 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.926752    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"]
Oct 30 22:02:21.930826 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:21.930799    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl"]
Oct 30 22:02:22.983546 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:22.983510    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:02:23.554333 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.554159    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:02:23.554333 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.554206    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:02:23.666113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.666082    5561 generic.go:356] "Generic (PLEG): container finished" podID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerID="c51f59f1aaad813a1bab35858caf2c678002c18ede6ae099bce13a9f10d8c8a2" exitCode=0
Oct 30 22:02:23.666279 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.666158    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" event={"ID":"05d6a011-281b-4179-bf8d-4954ecb3df4d","Type":"ContainerDied","Data":"c51f59f1aaad813a1bab35858caf2c678002c18ede6ae099bce13a9f10d8c8a2"}
Oct 30 22:02:23.810220 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.810145    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 22:02:23.847916 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.847884    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abc18d2e-6761-4564-8830-055e26806395" path="/var/lib/kubelet/pods/abc18d2e-6761-4564-8830-055e26806395/volumes"
Oct 30 22:02:23.849933 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.849914    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-token\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-token\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.849998 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.849976    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-data\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850034 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850021    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05d6a011-281b-4179-bf8d-4954ecb3df4d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d6a011-281b-4179-bf8d-4954ecb3df4d-istiod-ca-cert\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850144 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850128    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-socket\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850181 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850171    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-kube-api-access-mbvgd\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-kube-api-access-mbvgd\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850250    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-certs\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850286    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-data" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:23.850408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850340    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-envoy\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850393    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-podinfo\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850414    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-socket" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:23.850514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850434    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05d6a011-281b-4179-bf8d-4954ecb3df4d-istiod-ca-cert" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:23.850662 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850555    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-certs" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:23.850729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850661    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-credential-socket\") pod \"05d6a011-281b-4179-bf8d-4954ecb3df4d\" (UID: \"05d6a011-281b-4179-bf8d-4954ecb3df4d\") "
Oct 30 22:02:23.850940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.850919    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-credential-socket" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:23.851435 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.851413    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.851500 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.851442    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.851500 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.851459    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d6a011-281b-4179-bf8d-4954ecb3df4d-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.851500 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.851474    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.851500 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.851490    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.852197 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.852177    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-token" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:23.852557 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.852538    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-envoy" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:23.852887 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.852866    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-podinfo" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:23.852946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.852902    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-kube-api-access-mbvgd" pod "05d6a011-281b-4179-bf8d-4954ecb3df4d" (UID: "05d6a011-281b-4179-bf8d-4954ecb3df4d"). InnerVolumeSpecName "kube-api-access-mbvgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:23.953298 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.953264    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.953476 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.953300    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.953476 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.953318    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:23.953476 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:23.953335    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbvgd\" (UniqueName: \"kubernetes.io/projected/05d6a011-281b-4179-bf8d-4954ecb3df4d-kube-api-access-mbvgd\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.112158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.112129    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 22:02:24.156021 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.155992    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/23b089a3-2fc6-4068-ba97-21bf694af449-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b089a3-2fc6-4068-ba97-21bf694af449-istio-podinfo\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156190 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156032    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-envoy\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156190 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156132    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-data\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156190 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156157    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-kube-api-access-q4kp2\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-kube-api-access-q4kp2\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156241    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-certs\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156447 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156402    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-credential-socket\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156447 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156432    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-data" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.156561 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156465    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-socket\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156561 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156531    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23b089a3-2fc6-4068-ba97-21bf694af449-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b089a3-2fc6-4068-ba97-21bf694af449-istiod-ca-cert\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156669 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156581    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-certs" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.156669 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156659    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-istio-token\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-istio-token\") pod \"23b089a3-2fc6-4068-ba97-21bf694af449\" (UID: \"23b089a3-2fc6-4068-ba97-21bf694af449\") "
Oct 30 22:02:24.156779 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156664    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-credential-socket" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.156830 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156793    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-socket" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.156915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.156881    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23b089a3-2fc6-4068-ba97-21bf694af449-istiod-ca-cert" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.157511 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.157484    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b089a3-2fc6-4068-ba97-21bf694af449-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.157630 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.157515    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.157630 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.157531    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.157630 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.157547    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.157630 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.157561    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.158421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.158393    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-kube-api-access-q4kp2" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "kube-api-access-q4kp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.158542 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.158483    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23b089a3-2fc6-4068-ba97-21bf694af449-istio-podinfo" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.158608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.158569    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-envoy" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.158713 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.158700    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-istio-token" pod "23b089a3-2fc6-4068-ba97-21bf694af449" (UID: "23b089a3-2fc6-4068-ba97-21bf694af449"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.258842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.258812    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4kp2\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-kube-api-access-q4kp2\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.258842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.258841    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b089a3-2fc6-4068-ba97-21bf694af449-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.259029 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.258850    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b089a3-2fc6-4068-ba97-21bf694af449-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.259029 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.258859    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b089a3-2fc6-4068-ba97-21bf694af449-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.695338 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.695306    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"
Oct 30 22:02:24.695669 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.695329    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6" event={"ID":"05d6a011-281b-4179-bf8d-4954ecb3df4d","Type":"ContainerDied","Data":"ea5e9628bb61978372ad545b7cb483e6c84152075836c1bbe08d1c5bb8d39a76"}
Oct 30 22:02:24.695762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.695712    5561 scope.go:122] "RemoveContainer" containerID="c51f59f1aaad813a1bab35858caf2c678002c18ede6ae099bce13a9f10d8c8a2"
Oct 30 22:02:24.697304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.697275    5561 generic.go:356] "Generic (PLEG): container finished" podID="23b089a3-2fc6-4068-ba97-21bf694af449" containerID="15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da" exitCode=0
Oct 30 22:02:24.697418 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.697343    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"
Oct 30 22:02:24.697486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.697455    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" event={"ID":"23b089a3-2fc6-4068-ba97-21bf694af449","Type":"ContainerDied","Data":"15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da"}
Oct 30 22:02:24.697551 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.697498    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs" event={"ID":"23b089a3-2fc6-4068-ba97-21bf694af449","Type":"ContainerDied","Data":"b3f55e81a6240fd47a6cdf299f537f072defc8b8c1f379f4b622bcc3e4fb87bf"}
Oct 30 22:02:24.709617 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.709592    5561 scope.go:122] "RemoveContainer" containerID="15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da"
Oct 30 22:02:24.728749 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.728725    5561 scope.go:122] "RemoveContainer" containerID="15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da"
Oct 30 22:02:24.729090 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:24.729062    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da\": container with ID starting with 15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da not found: ID does not exist" containerID="15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da"
Oct 30 22:02:24.729195 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.729097    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da"} err="failed to get container status \"15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da\": rpc error: code = NotFound desc = could not find container \"15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da\": container with ID starting with 15335dbca8092c00db90501805e23757851d1dcbbabc183ba9aa10b6e7bc66da not found: ID does not exist"
Oct 30 22:02:24.932610 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.932561    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"]
Oct 30 22:02:24.970248 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:24.970210    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6"]
Oct 30 22:02:25.051422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:25.051342    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"]
Oct 30 22:02:25.055671 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:25.055636    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs"]
Oct 30 22:02:25.848234 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:25.848196    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" path="/var/lib/kubelet/pods/05d6a011-281b-4179-bf8d-4954ecb3df4d/volumes"
Oct 30 22:02:25.848808 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:25.848791    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" path="/var/lib/kubelet/pods/23b089a3-2fc6-4068-ba97-21bf694af449/volumes"
Oct 30 22:02:29.306365 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:29.306318    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"]
Oct 30 22:02:29.429065 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:29.428819    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"]
Oct 30 22:02:34.307224 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.307184    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m" podUID="de5d7920-c867-4deb-b358-b364ab788f64" containerName="payload-processing" containerID="cri-o://4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3" gracePeriod=25
Oct 30 22:02:34.429441 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.429391    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd" podUID="c55a47ba-c4fd-43e6-adfe-11e33d829cea" containerName="payload-pre-processing" containerID="cri-o://fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01" gracePeriod=25
Oct 30 22:02:34.582609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.582542    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:02:34.629369 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.629331    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de5d7920-c867-4deb-b358-b364ab788f64-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5d7920-c867-4deb-b358-b364ab788f64-plugins-config-volume\") pod \"de5d7920-c867-4deb-b358-b364ab788f64\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") "
Oct 30 22:02:34.629672 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.629652    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de5d7920-c867-4deb-b358-b364ab788f64-kube-api-access-dshsz\" (UniqueName: \"kubernetes.io/projected/de5d7920-c867-4deb-b358-b364ab788f64-kube-api-access-dshsz\") pod \"de5d7920-c867-4deb-b358-b364ab788f64\" (UID: \"de5d7920-c867-4deb-b358-b364ab788f64\") "
Oct 30 22:02:34.629781 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.629743    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de5d7920-c867-4deb-b358-b364ab788f64-plugins-config-volume" pod "de5d7920-c867-4deb-b358-b364ab788f64" (UID: "de5d7920-c867-4deb-b358-b364ab788f64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:34.630282 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.630260    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5d7920-c867-4deb-b358-b364ab788f64-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:34.632345 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.632319    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5d7920-c867-4deb-b358-b364ab788f64-kube-api-access-dshsz" pod "de5d7920-c867-4deb-b358-b364ab788f64" (UID: "de5d7920-c867-4deb-b358-b364ab788f64"). InnerVolumeSpecName "kube-api-access-dshsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:34.695085 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.695059    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:02:34.731392 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.731341    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c55a47ba-c4fd-43e6-adfe-11e33d829cea-kube-api-access-k7tc5\" (UniqueName: \"kubernetes.io/projected/c55a47ba-c4fd-43e6-adfe-11e33d829cea-kube-api-access-k7tc5\") pod \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") "
Oct 30 22:02:34.731582 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.731566    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c55a47ba-c4fd-43e6-adfe-11e33d829cea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55a47ba-c4fd-43e6-adfe-11e33d829cea-plugins-config-volume\") pod \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\" (UID: \"c55a47ba-c4fd-43e6-adfe-11e33d829cea\") "
Oct 30 22:02:34.731972 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.731947    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55a47ba-c4fd-43e6-adfe-11e33d829cea-plugins-config-volume" pod "c55a47ba-c4fd-43e6-adfe-11e33d829cea" (UID: "c55a47ba-c4fd-43e6-adfe-11e33d829cea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:34.732246 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.732228    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55a47ba-c4fd-43e6-adfe-11e33d829cea-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:34.732325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.732247    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dshsz\" (UniqueName: \"kubernetes.io/projected/de5d7920-c867-4deb-b358-b364ab788f64-kube-api-access-dshsz\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:34.733517 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.733489    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55a47ba-c4fd-43e6-adfe-11e33d829cea-kube-api-access-k7tc5" pod "c55a47ba-c4fd-43e6-adfe-11e33d829cea" (UID: "c55a47ba-c4fd-43e6-adfe-11e33d829cea"). InnerVolumeSpecName "kube-api-access-k7tc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:34.833816 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:34.833731    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7tc5\" (UniqueName: \"kubernetes.io/projected/c55a47ba-c4fd-43e6-adfe-11e33d829cea-kube-api-access-k7tc5\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:35.126494 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.126462    5561 generic.go:356] "Generic (PLEG): container finished" podID="c55a47ba-c4fd-43e6-adfe-11e33d829cea" containerID="fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01" exitCode=0
Oct 30 22:02:35.126652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.126505    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"
Oct 30 22:02:35.126652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.126550    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd" event={"ID":"c55a47ba-c4fd-43e6-adfe-11e33d829cea","Type":"ContainerDied","Data":"fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01"}
Oct 30 22:02:35.126652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.126598    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd" event={"ID":"c55a47ba-c4fd-43e6-adfe-11e33d829cea","Type":"ContainerDied","Data":"b7800ac5053c57bd25cb48bb4aca7101ff61bf8b5a605883e09d052d8cd5622f"}
Oct 30 22:02:35.126652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.126628    5561 scope.go:122] "RemoveContainer" containerID="fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01"
Oct 30 22:02:35.127949 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.127926    5561 generic.go:356] "Generic (PLEG): container finished" podID="de5d7920-c867-4deb-b358-b364ab788f64" containerID="4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3" exitCode=0
Oct 30 22:02:35.128055 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.127963    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m" event={"ID":"de5d7920-c867-4deb-b358-b364ab788f64","Type":"ContainerDied","Data":"4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3"}
Oct 30 22:02:35.128055 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.127976    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"
Oct 30 22:02:35.128055 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.127993    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m" event={"ID":"de5d7920-c867-4deb-b358-b364ab788f64","Type":"ContainerDied","Data":"d7432c070e8c214a373388db1fa1ef29ad1e674e0eb8470b09cdd193407441ea"}
Oct 30 22:02:35.139173 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.139156    5561 scope.go:122] "RemoveContainer" containerID="fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01"
Oct 30 22:02:35.139486 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:35.139467    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01\": container with ID starting with fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01 not found: ID does not exist" containerID="fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01"
Oct 30 22:02:35.139560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.139492    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01"} err="failed to get container status \"fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01\": rpc error: code = NotFound desc = could not find container \"fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01\": container with ID starting with fa4cd1c828a6298d4ae7bd0138e8a1e3fe0e1acfcd8b6e5204dc1ada1418bb01 not found: ID does not exist"
Oct 30 22:02:35.139560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.139508    5561 scope.go:122] "RemoveContainer" containerID="4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3"
Oct 30 22:02:35.157497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.157369    5561 scope.go:122] "RemoveContainer" containerID="4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3"
Oct 30 22:02:35.165832 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:35.165704    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3\": container with ID starting with 4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3 not found: ID does not exist" containerID="4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3"
Oct 30 22:02:35.165832 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.165767    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3"} err="failed to get container status \"4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3\": rpc error: code = NotFound desc = could not find container \"4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3\": container with ID starting with 4ec2546360c0646027cc1daeed43e82d1e12c7ca2e913267c9608c278e8dccd3 not found: ID does not exist"
Oct 30 22:02:35.338380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.334798    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"]
Oct 30 22:02:35.446476 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.433134    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd"]
Oct 30 22:02:35.463466 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.463429    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"]
Oct 30 22:02:35.465737 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.465711    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m"]
Oct 30 22:02:35.847467 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.847387    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c55a47ba-c4fd-43e6-adfe-11e33d829cea" path="/var/lib/kubelet/pods/c55a47ba-c4fd-43e6-adfe-11e33d829cea/volumes"
Oct 30 22:02:35.847888 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:35.847873    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de5d7920-c867-4deb-b358-b364ab788f64" path="/var/lib/kubelet/pods/de5d7920-c867-4deb-b358-b364ab788f64/volumes"
Oct 30 22:02:45.479996 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:45.479953    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"]
Oct 30 22:02:45.480444 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:45.480250    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy" containerID="cri-o://417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58" gracePeriod=30
Oct 30 22:02:50.760334 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.760266    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 22:02:50.894545 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894517    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istiod-ca-cert\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.894694 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894550    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-token\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-token\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.894694 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894587    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-certs\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.894797 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894699    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-podinfo\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.894797 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894774    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-credential-socket\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.894906 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894845    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-kube-api-access-9hhp5\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-kube-api-access-9hhp5\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.894906 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894858    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-certs" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:50.895027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894893    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istiod-ca-cert" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:50.895027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.894902    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-envoy\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.895154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895067    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-credential-socket" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:50.895154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895108    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-socket\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.895249 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895159    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-data\") pod \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\" (UID: \"2fc08696-54ad-47eb-9f3c-07fe96105a8b\") "
Oct 30 22:02:50.895498 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895477    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-socket" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:50.895603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895580    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-data" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:50.895963 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895938    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.896062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895971    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.896062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.895987    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.896062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.896001    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.896062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.896013    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.897049 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.897028    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-podinfo" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:50.897143 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.897084    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-token" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:50.897341 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.897327    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-envoy" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:50.897435 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.897416    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-kube-api-access-9hhp5" pod "2fc08696-54ad-47eb-9f3c-07fe96105a8b" (UID: "2fc08696-54ad-47eb-9f3c-07fe96105a8b"). InnerVolumeSpecName "kube-api-access-9hhp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:50.997952 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.997924    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.997952 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.997949    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.997952 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.997959    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hhp5\" (UniqueName: \"kubernetes.io/projected/2fc08696-54ad-47eb-9f3c-07fe96105a8b-kube-api-access-9hhp5\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:50.998172 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:50.997969    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fc08696-54ad-47eb-9f3c-07fe96105a8b-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:51.310286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.310254    5561 generic.go:356] "Generic (PLEG): container finished" podID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerID="417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58" exitCode=0
Oct 30 22:02:51.310478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.310303    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"
Oct 30 22:02:51.310478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.310329    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" event={"ID":"2fc08696-54ad-47eb-9f3c-07fe96105a8b","Type":"ContainerDied","Data":"417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58"}
Oct 30 22:02:51.310478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.310401    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77" event={"ID":"2fc08696-54ad-47eb-9f3c-07fe96105a8b","Type":"ContainerDied","Data":"8c4b0a83a2bb4e2386b4beb6325d21cc882e39b2325bce5f1fa0175842123fde"}
Oct 30 22:02:51.310478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.310422    5561 scope.go:122] "RemoveContainer" containerID="417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58"
Oct 30 22:02:51.321735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.321718    5561 scope.go:122] "RemoveContainer" containerID="417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58"
Oct 30 22:02:51.322020 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:51.322000    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58\": container with ID starting with 417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58 not found: ID does not exist" containerID="417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58"
Oct 30 22:02:51.322095 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.322027    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58"} err="failed to get container status \"417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58\": rpc error: code = NotFound desc = could not find container \"417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58\": container with ID starting with 417961aad3207baa3a56cafce685f19926a31ef431332ff586832578c65f2a58 not found: ID does not exist"
Oct 30 22:02:51.479054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.479028    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"]
Oct 30 22:02:51.482202 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.482183    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77"]
Oct 30 22:02:51.847187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:51.847118    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" path="/var/lib/kubelet/pods/2fc08696-54ad-47eb-9f3c-07fe96105a8b/volumes"
Oct 30 22:02:52.984416 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:52.984376    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:02:57.757803 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:57.757767    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"]
Oct 30 22:02:57.758268 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:57.757992    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4" podUID="9cdd3026-6800-40ab-989f-7c336fe9bf86" containerName="maas-api" containerID="cri-o://56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6" gracePeriod=30
Oct 30 22:02:58.261742 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.253723    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:02:58.333390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.331103    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9cdd3026-6800-40ab-989f-7c336fe9bf86-kube-api-access-nz4qh\" (UniqueName: \"kubernetes.io/projected/9cdd3026-6800-40ab-989f-7c336fe9bf86-kube-api-access-nz4qh\") pod \"9cdd3026-6800-40ab-989f-7c336fe9bf86\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") "
Oct 30 22:02:58.333390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.331259    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls\") pod \"9cdd3026-6800-40ab-989f-7c336fe9bf86\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") "
Oct 30 22:02:58.333390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.331405    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-metrics-certs\") pod \"9cdd3026-6800-40ab-989f-7c336fe9bf86\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") "
Oct 30 22:02:58.333390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.331485    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9cdd3026-6800-40ab-989f-7c336fe9bf86-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdd3026-6800-40ab-989f-7c336fe9bf86-openshift-service-ca\") pod \"9cdd3026-6800-40ab-989f-7c336fe9bf86\" (UID: \"9cdd3026-6800-40ab-989f-7c336fe9bf86\") "
Oct 30 22:02:58.354375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.350570    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cdd3026-6800-40ab-989f-7c336fe9bf86-openshift-service-ca" pod "9cdd3026-6800-40ab-989f-7c336fe9bf86" (UID: "9cdd3026-6800-40ab-989f-7c336fe9bf86"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:58.354375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.353332    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cdd3026-6800-40ab-989f-7c336fe9bf86-kube-api-access-nz4qh" pod "9cdd3026-6800-40ab-989f-7c336fe9bf86" (UID: "9cdd3026-6800-40ab-989f-7c336fe9bf86"). InnerVolumeSpecName "kube-api-access-nz4qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:58.431805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.431656    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls" pod "9cdd3026-6800-40ab-989f-7c336fe9bf86" (UID: "9cdd3026-6800-40ab-989f-7c336fe9bf86"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:58.432170 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.432139    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-metrics-certs" pod "9cdd3026-6800-40ab-989f-7c336fe9bf86" (UID: "9cdd3026-6800-40ab-989f-7c336fe9bf86"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:58.435431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.435400    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:58.435568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.435435    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdd3026-6800-40ab-989f-7c336fe9bf86-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:58.435568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.435452    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdd3026-6800-40ab-989f-7c336fe9bf86-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:58.435568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.435468    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz4qh\" (UniqueName: \"kubernetes.io/projected/9cdd3026-6800-40ab-989f-7c336fe9bf86-kube-api-access-nz4qh\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:58.437215 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.437182    5561 generic.go:356] "Generic (PLEG): container finished" podID="9cdd3026-6800-40ab-989f-7c336fe9bf86" containerID="56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6" exitCode=0
Oct 30 22:02:58.437319 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.437261    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"
Oct 30 22:02:58.437458 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.437423    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4" event={"ID":"9cdd3026-6800-40ab-989f-7c336fe9bf86","Type":"ContainerDied","Data":"56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6"}
Oct 30 22:02:58.437561 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.437516    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4" event={"ID":"9cdd3026-6800-40ab-989f-7c336fe9bf86","Type":"ContainerDied","Data":"9e88f0f676e4b1046e29f095ee1110ccc2db12627337373062ca10b1173d28c9"}
Oct 30 22:02:58.437561 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.437546    5561 scope.go:122] "RemoveContainer" containerID="56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6"
Oct 30 22:02:58.457068 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.457049    5561 scope.go:122] "RemoveContainer" containerID="56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6"
Oct 30 22:02:58.457428 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:02:58.457404    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6\": container with ID starting with 56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6 not found: ID does not exist" containerID="56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6"
Oct 30 22:02:58.457551 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.457433    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6"} err="failed to get container status \"56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6\": rpc error: code = NotFound desc = could not find container \"56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6\": container with ID starting with 56c4d2c603aefadfb29dc7811670055571ac1b7cd6cab9fbe0306550694477a6 not found: ID does not exist"
Oct 30 22:02:58.619695 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.619662    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"]
Oct 30 22:02:58.623460 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:58.623433    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b8311b7-765667c68-7vrv4"]
Oct 30 22:02:59.884377 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:02:59.881058    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9cdd3026-6800-40ab-989f-7c336fe9bf86" path="/var/lib/kubelet/pods/9cdd3026-6800-40ab-989f-7c336fe9bf86/volumes"
Oct 30 22:03:00.195380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:00.195269    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"]
Oct 30 22:03:00.454786 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:00.454679    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy" containerID="cri-o://e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c" gracePeriod=30
Oct 30 22:03:04.404417 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.404379    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"]
Oct 30 22:03:04.404922 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.404604    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll" podUID="480dee01-cb09-401a-8ca5-7daeee0251dd" containerName="maas-api" containerID="cri-o://8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88" gracePeriod=30
Oct 30 22:03:04.833458 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.833419    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:03:04.953370 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.950471    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-metrics-certs\") pod \"480dee01-cb09-401a-8ca5-7daeee0251dd\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") "
Oct 30 22:03:04.953370 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.951042    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/480dee01-cb09-401a-8ca5-7daeee0251dd-kube-api-access-bnl4g\" (UniqueName: \"kubernetes.io/projected/480dee01-cb09-401a-8ca5-7daeee0251dd-kube-api-access-bnl4g\") pod \"480dee01-cb09-401a-8ca5-7daeee0251dd\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") "
Oct 30 22:03:04.953370 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.951468    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls\") pod \"480dee01-cb09-401a-8ca5-7daeee0251dd\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") "
Oct 30 22:03:04.953370 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.951710    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/480dee01-cb09-401a-8ca5-7daeee0251dd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480dee01-cb09-401a-8ca5-7daeee0251dd-openshift-service-ca\") pod \"480dee01-cb09-401a-8ca5-7daeee0251dd\" (UID: \"480dee01-cb09-401a-8ca5-7daeee0251dd\") "
Oct 30 22:03:04.953370 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.953303    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/480dee01-cb09-401a-8ca5-7daeee0251dd-openshift-service-ca" pod "480dee01-cb09-401a-8ca5-7daeee0251dd" (UID: "480dee01-cb09-401a-8ca5-7daeee0251dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:04.963110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.958916    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480dee01-cb09-401a-8ca5-7daeee0251dd-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:04.980369 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.965216    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480dee01-cb09-401a-8ca5-7daeee0251dd-kube-api-access-bnl4g" pod "480dee01-cb09-401a-8ca5-7daeee0251dd" (UID: "480dee01-cb09-401a-8ca5-7daeee0251dd"). InnerVolumeSpecName "kube-api-access-bnl4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:04.980369 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.965510    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-metrics-certs" pod "480dee01-cb09-401a-8ca5-7daeee0251dd" (UID: "480dee01-cb09-401a-8ca5-7daeee0251dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:04.980369 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:04.967085    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls" pod "480dee01-cb09-401a-8ca5-7daeee0251dd" (UID: "480dee01-cb09-401a-8ca5-7daeee0251dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:05.060015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.059932    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.060015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.059962    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnl4g\" (UniqueName: \"kubernetes.io/projected/480dee01-cb09-401a-8ca5-7daeee0251dd-kube-api-access-bnl4g\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.060015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.059975    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480dee01-cb09-401a-8ca5-7daeee0251dd-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.494261 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.494228    5561 generic.go:356] "Generic (PLEG): container finished" podID="480dee01-cb09-401a-8ca5-7daeee0251dd" containerID="8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88" exitCode=0
Oct 30 22:03:05.494607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.494276    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"
Oct 30 22:03:05.494607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.494311    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll" event={"ID":"480dee01-cb09-401a-8ca5-7daeee0251dd","Type":"ContainerDied","Data":"8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88"}
Oct 30 22:03:05.494607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.494391    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll" event={"ID":"480dee01-cb09-401a-8ca5-7daeee0251dd","Type":"ContainerDied","Data":"d61ed9c0e2aab0677950ec6f58851cc212ae002d72bf856bb124dffaf9be1cb6"}
Oct 30 22:03:05.494607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.494417    5561 scope.go:122] "RemoveContainer" containerID="8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88"
Oct 30 22:03:05.567920 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.567900    5561 scope.go:122] "RemoveContainer" containerID="8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88"
Oct 30 22:03:05.568245 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:03:05.568219    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88\": container with ID starting with 8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88 not found: ID does not exist" containerID="8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88"
Oct 30 22:03:05.568372 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.568249    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88"} err="failed to get container status \"8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88\": rpc error: code = NotFound desc = could not find container \"8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88\": container with ID starting with 8d2a5dc4b33554d48371f7e38800a7fa2dc26d4fa67ff5fabd9463fd45978e88 not found: ID does not exist"
Oct 30 22:03:05.664386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.664318    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"]
Oct 30 22:03:05.712608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.712571    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2c278-749988bbf8-q9cll"]
Oct 30 22:03:05.747932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.747907    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:03:05.847296 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.847263    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="480dee01-cb09-401a-8ca5-7daeee0251dd" path="/var/lib/kubelet/pods/480dee01-cb09-401a-8ca5-7daeee0251dd/volumes"
Oct 30 22:03:05.871800 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.871771    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-envoy\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.871983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.871817    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-istio-token\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-istio-token\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.871983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.871865    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/70638018-48d1-4ef3-b021-b974ee34d87a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70638018-48d1-4ef3-b021-b974ee34d87a-istio-podinfo\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.871983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.871888    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70638018-48d1-4ef3-b021-b974ee34d87a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70638018-48d1-4ef3-b021-b974ee34d87a-istiod-ca-cert\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.871983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.871922    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-socket\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.872206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.871995    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-credential-socket\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.872206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872035    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-data\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.872206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872076    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-certs\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.872206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872162    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-kube-api-access-cs2hd\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-kube-api-access-cs2hd\") pod \"70638018-48d1-4ef3-b021-b974ee34d87a\" (UID: \"70638018-48d1-4ef3-b021-b974ee34d87a\") "
Oct 30 22:03:05.872486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872401    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-credential-socket" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:05.872486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872476    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-socket" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:05.872680 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872659    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-certs" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:05.872747 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872676    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-data" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:05.872866 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.872845    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70638018-48d1-4ef3-b021-b974ee34d87a-istiod-ca-cert" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:05.873102 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.873081    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70638018-48d1-4ef3-b021-b974ee34d87a-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.873164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.873108    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.873164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.873124    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.873164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.873137    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.873164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.873150    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.874170 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.874141    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-envoy" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:05.874643 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.874620    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-istio-token" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:05.874944 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.874918    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-kube-api-access-cs2hd" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "kube-api-access-cs2hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:05.875108 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.875083    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70638018-48d1-4ef3-b021-b974ee34d87a-istio-podinfo" pod "70638018-48d1-4ef3-b021-b974ee34d87a" (UID: "70638018-48d1-4ef3-b021-b974ee34d87a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:05.974846 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.974791    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs2hd\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-kube-api-access-cs2hd\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.974846 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.974825    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70638018-48d1-4ef3-b021-b974ee34d87a-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.974846 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.974840    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70638018-48d1-4ef3-b021-b974ee34d87a-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:05.975133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:05.974864    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70638018-48d1-4ef3-b021-b974ee34d87a-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:06.547375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.540243    5561 generic.go:356] "Generic (PLEG): container finished" podID="70638018-48d1-4ef3-b021-b974ee34d87a" containerID="e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c" exitCode=0
Oct 30 22:03:06.547375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.540520    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" event={"ID":"70638018-48d1-4ef3-b021-b974ee34d87a","Type":"ContainerDied","Data":"e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c"}
Oct 30 22:03:06.547375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.540581    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf" event={"ID":"70638018-48d1-4ef3-b021-b974ee34d87a","Type":"ContainerDied","Data":"7efc8db1ab32e32a51687c6f849757a0e9b4dc520eb2baa5de1b2fa0ff613ce8"}
Oct 30 22:03:06.547375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.540624    5561 scope.go:122] "RemoveContainer" containerID="e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c"
Oct 30 22:03:06.547375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.540736    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"
Oct 30 22:03:06.609707 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.609571    5561 scope.go:122] "RemoveContainer" containerID="77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e"
Oct 30 22:03:06.628953 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.628736    5561 scope.go:122] "RemoveContainer" containerID="e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c"
Oct 30 22:03:06.629201 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:03:06.629086    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c\": container with ID starting with e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c not found: ID does not exist" containerID="e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c"
Oct 30 22:03:06.629201 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.629122    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c"} err="failed to get container status \"e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c\": rpc error: code = NotFound desc = could not find container \"e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c\": container with ID starting with e92d3015eb71206875e44e5fd4a53976345051e00b005eb4b3f4cfa34e708a8c not found: ID does not exist"
Oct 30 22:03:06.629201 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.629141    5561 scope.go:122] "RemoveContainer" containerID="77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e"
Oct 30 22:03:06.629694 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:03:06.629640    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e\": container with ID starting with 77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e not found: ID does not exist" containerID="77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e"
Oct 30 22:03:06.629694 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.629670    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e"} err="failed to get container status \"77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e\": rpc error: code = NotFound desc = could not find container \"77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e\": container with ID starting with 77b6a66de1cb58aec50fbcb9327c8dd0a6db8a46e39cd888e7558c006c01c87e not found: ID does not exist"
Oct 30 22:03:06.720472 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.720442    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"]
Oct 30 22:03:06.778411 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.775184    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf"]
Oct 30 22:03:06.780152 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.780109    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"]
Oct 30 22:03:06.780421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:06.780371    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy" containerID="cri-o://9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3" gracePeriod=30
Oct 30 22:03:07.848406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:07.848372    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" path="/var/lib/kubelet/pods/70638018-48d1-4ef3-b021-b974ee34d87a/volumes"
Oct 30 22:03:08.090587 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.090508    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"]
Oct 30 22:03:08.091455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091438    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9cdd3026-6800-40ab-989f-7c336fe9bf86" containerName="maas-api"
Oct 30 22:03:08.091510 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091458    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9cdd3026-6800-40ab-989f-7c336fe9bf86" containerName="maas-api"
Oct 30 22:03:08.091510 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091480    5561 cpu_manager.go:414] "containerMap: removing container" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy"
Oct 30 22:03:08.091510 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091488    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy"
Oct 30 22:03:08.091510 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091501    5561 cpu_manager.go:414] "containerMap: removing container" podUID="c55a47ba-c4fd-43e6-adfe-11e33d829cea" containerName="payload-pre-processing"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091509    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c55a47ba-c4fd-43e6-adfe-11e33d829cea" containerName="payload-pre-processing"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091531    5561 cpu_manager.go:414] "containerMap: removing container" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091539    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091558    5561 cpu_manager.go:414] "containerMap: removing container" podUID="abc18d2e-6761-4564-8830-055e26806395" containerName="payload-processing"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091565    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abc18d2e-6761-4564-8830-055e26806395" containerName="payload-processing"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091582    5561 cpu_manager.go:414] "containerMap: removing container" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091589    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091600    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a8031899-312a-45c2-9d82-814ffa30e5c0" containerName="payload-pre-processing"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091607    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8031899-312a-45c2-9d82-814ffa30e5c0" containerName="payload-pre-processing"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091618    5561 cpu_manager.go:414] "containerMap: removing container" podUID="de5d7920-c867-4deb-b358-b364ab788f64" containerName="payload-processing"
Oct 30 22:03:08.091635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091627    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de5d7920-c867-4deb-b358-b364ab788f64" containerName="payload-processing"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091643    5561 cpu_manager.go:414] "containerMap: removing container" podUID="480dee01-cb09-401a-8ca5-7daeee0251dd" containerName="maas-api"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091651    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="480dee01-cb09-401a-8ca5-7daeee0251dd" containerName="maas-api"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091668    5561 cpu_manager.go:414] "containerMap: removing container" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091675    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091704    5561 cpu_manager.go:414] "containerMap: removing container" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091711    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091839    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091855    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="70638018-48d1-4ef3-b021-b974ee34d87a" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091866    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9cdd3026-6800-40ab-989f-7c336fe9bf86" containerName="maas-api"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091876    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fc08696-54ad-47eb-9f3c-07fe96105a8b" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091886    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="c55a47ba-c4fd-43e6-adfe-11e33d829cea" containerName="payload-pre-processing"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091897    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="23b089a3-2fc6-4068-ba97-21bf694af449" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091905    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="abc18d2e-6761-4564-8830-055e26806395" containerName="payload-processing"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091917    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="05d6a011-281b-4179-bf8d-4954ecb3df4d" containerName="istio-proxy"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091927    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8031899-312a-45c2-9d82-814ffa30e5c0" containerName="payload-pre-processing"
Oct 30 22:03:08.091934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091936    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="de5d7920-c867-4deb-b358-b364ab788f64" containerName="payload-processing"
Oct 30 22:03:08.092431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.091944    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="480dee01-cb09-401a-8ca5-7daeee0251dd" containerName="maas-api"
Oct 30 22:03:08.098462 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.098440    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.170793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.170758    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"]
Oct 30 22:03:08.202539 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.202497    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w6-9d4043-openshift-default-dockercfg-m5fnk\""
Oct 30 22:03:08.285396 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.284989    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-certs\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285396 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285048    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-data\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285396 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285173    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-token\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285396 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285283    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftdlm\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-kube-api-access-ftdlm\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285396 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285392    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ea8511-f145-4af1-95bc-6e10dc90379c-istiod-ca-cert\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285802 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285434    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-socket\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285802 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285533    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-podinfo\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285802 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285648    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-envoy\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.285802 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.285690    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-credential-socket\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.404783    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-podinfo\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.404932    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-envoy\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.404975    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-credential-socket\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.405044    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-certs\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.405070    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-data\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.405155    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-token\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.405276    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdlm\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-kube-api-access-ftdlm\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.405387    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ea8511-f145-4af1-95bc-6e10dc90379c-istiod-ca-cert\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.405439    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-socket\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.411371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.406036    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-socket\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.417765 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.417733    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-data\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.420950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.420921    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-envoy\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.421123 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.420963    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-credential-socket\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.421226 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.421198    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-certs\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.441500 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.422461    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ea8511-f145-4af1-95bc-6e10dc90379c-istiod-ca-cert\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.463317 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.463278    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-podinfo\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.502176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.498444    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-token\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.508262 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.508229    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdlm\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-kube-api-access-ftdlm\") pod \"e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") " pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:08.712171 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:08.712092    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:09.162289 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:03:09.161767    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ea8511_f145_4af1_95bc_6e10dc90379c.slice/crio-68804c84d54b9bafe2c6c12cfbdff3362122826cb4efb983522a9d4c66627a75 WatchSource:0}: Error finding container 68804c84d54b9bafe2c6c12cfbdff3362122826cb4efb983522a9d4c66627a75: Status 404 returned error can't find the container with id 68804c84d54b9bafe2c6c12cfbdff3362122826cb4efb983522a9d4c66627a75
Oct 30 22:03:09.162289 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.162236    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"]
Oct 30 22:03:09.164751 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.164711    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:09.164874 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.164792    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:09.164874 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.164839    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:09.638446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.638406    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" event={"ID":"24ea8511-f145-4af1-95bc-6e10dc90379c","Type":"ContainerStarted","Data":"3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8"}
Oct 30 22:03:09.638633 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.638479    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" event={"ID":"24ea8511-f145-4af1-95bc-6e10dc90379c","Type":"ContainerStarted","Data":"68804c84d54b9bafe2c6c12cfbdff3362122826cb4efb983522a9d4c66627a75"}
Oct 30 22:03:09.712489 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.712459    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:09.759530 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:09.759486    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podStartSLOduration=1.759474082 podStartE2EDuration="1.759474082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:09.757787094 +0000 UTC m=+1738.035480565" watchObservedRunningTime="2026-10-30 22:03:09.759474082 +0000 UTC m=+1738.037167553"
Oct 30 22:03:10.717882 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:10.717849    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:11.960905 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:11.960859    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:11.961407 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:11.960936    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:03:12.146827 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.146795    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:03:12.149660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.149632    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-kube-api-access-2glnp\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-kube-api-access-2glnp\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.149867 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.149834    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-envoy\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.149916    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cfb83f97-7336-40d1-a749-511d9118b63d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb83f97-7336-40d1-a749-511d9118b63d-istio-podinfo\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.149969    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-certs\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.150042    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-data\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.150088    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-istio-token\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-istio-token\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.150130    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-credential-socket\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.150658    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-certs" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.150666    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-socket\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.150769    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfb83f97-7336-40d1-a749-511d9118b63d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb83f97-7336-40d1-a749-511d9118b63d-istiod-ca-cert\") pod \"cfb83f97-7336-40d1-a749-511d9118b63d\" (UID: \"cfb83f97-7336-40d1-a749-511d9118b63d\") "
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.150880    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-socket" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.151055    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-data" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.151597    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.151619    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.151635    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.151807    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-credential-socket" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:12.152568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.151908    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfb83f97-7336-40d1-a749-511d9118b63d-istiod-ca-cert" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:12.157292 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.153648    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-istio-token" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:12.176278 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.176244    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfb83f97-7336-40d1-a749-511d9118b63d-istio-podinfo" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:12.176414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.176322    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-kube-api-access-2glnp" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "kube-api-access-2glnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:12.177602 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.177581    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-envoy" pod "cfb83f97-7336-40d1-a749-511d9118b63d" (UID: "cfb83f97-7336-40d1-a749-511d9118b63d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:12.257408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.257383    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.257556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.257413    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.257556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.257431    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfb83f97-7336-40d1-a749-511d9118b63d-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.257556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.257448    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2glnp\" (UniqueName: \"kubernetes.io/projected/cfb83f97-7336-40d1-a749-511d9118b63d-kube-api-access-2glnp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.257556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.257463    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfb83f97-7336-40d1-a749-511d9118b63d-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.257556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.257476    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfb83f97-7336-40d1-a749-511d9118b63d-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:12.832682 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.832649    5561 generic.go:356] "Generic (PLEG): container finished" podID="cfb83f97-7336-40d1-a749-511d9118b63d" containerID="9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3" exitCode=0
Oct 30 22:03:12.832844 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.832697    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"
Oct 30 22:03:12.832844 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.832738    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" event={"ID":"cfb83f97-7336-40d1-a749-511d9118b63d","Type":"ContainerDied","Data":"9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3"}
Oct 30 22:03:12.832844 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.832798    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg" event={"ID":"cfb83f97-7336-40d1-a749-511d9118b63d","Type":"ContainerDied","Data":"a5d91cb63ac0657ed2623305ac0b0794b8f4d0f638a6447ef4bad4783c4fb99a"}
Oct 30 22:03:12.832844 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.832826    5561 scope.go:122] "RemoveContainer" containerID="9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3"
Oct 30 22:03:12.843733 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.843718    5561 scope.go:122] "RemoveContainer" containerID="f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3"
Oct 30 22:03:12.859324 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.859309    5561 scope.go:122] "RemoveContainer" containerID="9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3"
Oct 30 22:03:12.859643 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:03:12.859623    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3\": container with ID starting with 9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3 not found: ID does not exist" containerID="9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3"
Oct 30 22:03:12.859694 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.859651    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3"} err="failed to get container status \"9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3\": rpc error: code = NotFound desc = could not find container \"9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3\": container with ID starting with 9d643cb2e7d49aab6ba001d1acb35806d2174ab7ab84247f1413b10488a097b3 not found: ID does not exist"
Oct 30 22:03:12.859694 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.859667    5561 scope.go:122] "RemoveContainer" containerID="f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3"
Oct 30 22:03:12.859885 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:03:12.859866    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3\": container with ID starting with f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3 not found: ID does not exist" containerID="f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3"
Oct 30 22:03:12.859922 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:12.859891    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3"} err="failed to get container status \"f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3\": rpc error: code = NotFound desc = could not find container \"f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3\": container with ID starting with f55906a1171237970d44a891658caf15b5f44f374921ae7420c1ecdc3df93be3 not found: ID does not exist"
Oct 30 22:03:13.021087 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:13.021057    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"]
Oct 30 22:03:13.071313 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:13.071275    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg"]
Oct 30 22:03:13.846096 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:13.846061    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" path="/var/lib/kubelet/pods/cfb83f97-7336-40d1-a749-511d9118b63d/volumes"
Oct 30 22:03:16.953974 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:16.953917    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"]
Oct 30 22:03:16.955710 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:16.955686    5561 cpu_manager.go:414] "containerMap: removing container" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy"
Oct 30 22:03:16.955843 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:16.955830    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy"
Oct 30 22:03:16.955981 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:16.955972    5561 cpu_manager.go:414] "containerMap: removing container" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy"
Oct 30 22:03:16.956056 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:16.956046    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy"
Oct 30 22:03:16.956317 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:16.956304    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy"
Oct 30 22:03:16.975376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:16.967192    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.027773 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.027729    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhdx8\" (UniqueName: \"kubernetes.io/projected/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-kube-api-access-mhdx8\") pod \"payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.028086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.027964    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.031100 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.031069    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"]
Oct 30 22:03:17.074573 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.067627    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w6-9d4043\""
Oct 30 22:03:17.074573 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.068316    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w6-9d4043-dockercfg-rgqbr\""
Oct 30 22:03:17.190857 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.190815    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhdx8\" (UniqueName: \"kubernetes.io/projected/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-kube-api-access-mhdx8\") pod \"payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.191050 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.191000    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.193376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.192053    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.227505 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.227420    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"]
Oct 30 22:03:17.252404 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.239391    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfb83f97-7336-40d1-a749-511d9118b63d" containerName="istio-proxy"
Oct 30 22:03:17.342159 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.336096    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhdx8\" (UniqueName: \"kubernetes.io/projected/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-kube-api-access-mhdx8\") pod \"payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.351632 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.351601    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"]
Oct 30 22:03:17.364482 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.352645    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.506024 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.505942    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tx9j\" (UniqueName: \"kubernetes.io/projected/729e775b-1219-4eb8-a091-afe81bb48455-kube-api-access-9tx9j\") pod \"payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.506024 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.505980    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729e775b-1219-4eb8-a091-afe81bb48455-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.608430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.608396    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tx9j\" (UniqueName: \"kubernetes.io/projected/729e775b-1219-4eb8-a091-afe81bb48455-kube-api-access-9tx9j\") pod \"payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.608640 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.608437    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729e775b-1219-4eb8-a091-afe81bb48455-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.609180 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.609155    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729e775b-1219-4eb8-a091-afe81bb48455-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.614032 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.614011    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:17.672392 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.672342    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tx9j\" (UniqueName: \"kubernetes.io/projected/729e775b-1219-4eb8-a091-afe81bb48455-kube-api-access-9tx9j\") pod \"payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.725163 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.725135    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:17.996716 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:03:17.996690    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f86ca2_a523_428c_b6ad_57c1a63ec4dc.slice/crio-f5160c5175d4f25862558cfe4c07d232e31d8ed4c565edfc1a301e1548b0b4cf WatchSource:0}: Error finding container f5160c5175d4f25862558cfe4c07d232e31d8ed4c565edfc1a301e1548b0b4cf: Status 404 returned error can't find the container with id f5160c5175d4f25862558cfe4c07d232e31d8ed4c565edfc1a301e1548b0b4cf
Oct 30 22:03:17.997761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:17.997738    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"]
Oct 30 22:03:18.109006 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:18.108983    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"]
Oct 30 22:03:18.109816 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:03:18.109795    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729e775b_1219_4eb8_a091_afe81bb48455.slice/crio-98b045083f2451de55372a3c8874e016c055725c875e15c7f011a9f8bf6cf7ba WatchSource:0}: Error finding container 98b045083f2451de55372a3c8874e016c055725c875e15c7f011a9f8bf6cf7ba: Status 404 returned error can't find the container with id 98b045083f2451de55372a3c8874e016c055725c875e15c7f011a9f8bf6cf7ba
Oct 30 22:03:18.934277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:18.934245    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t" event={"ID":"729e775b-1219-4eb8-a091-afe81bb48455","Type":"ContainerStarted","Data":"9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a"}
Oct 30 22:03:18.934479 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:18.934306    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t" event={"ID":"729e775b-1219-4eb8-a091-afe81bb48455","Type":"ContainerStarted","Data":"98b045083f2451de55372a3c8874e016c055725c875e15c7f011a9f8bf6cf7ba"}
Oct 30 22:03:18.934479 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:18.934336    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:18.936266 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:18.936233    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n" event={"ID":"88f86ca2-a523-428c-b6ad-57c1a63ec4dc","Type":"ContainerStarted","Data":"fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30"}
Oct 30 22:03:18.936417 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:18.936284    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:18.936417 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:18.936299    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n" event={"ID":"88f86ca2-a523-428c-b6ad-57c1a63ec4dc","Type":"ContainerStarted","Data":"f5160c5175d4f25862558cfe4c07d232e31d8ed4c565edfc1a301e1548b0b4cf"}
Oct 30 22:03:19.053851 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:19.053802    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t" podStartSLOduration=2.053789754 podStartE2EDuration="2.053789754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:19.050334744 +0000 UTC m=+1747.328028216" watchObservedRunningTime="2026-10-30 22:03:19.053789754 +0000 UTC m=+1747.331483261"
Oct 30 22:03:19.193424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:19.192129    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n" podStartSLOduration=3.192105421 podStartE2EDuration="3.192105421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:19.188843193 +0000 UTC m=+1747.466536672" watchObservedRunningTime="2026-10-30 22:03:19.192105421 +0000 UTC m=+1747.469798893"
Oct 30 22:03:23.103399 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:03:23.100197    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:03:23.870797 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:23.870764    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:03:29.972219 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:29.972171    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:03:29.972889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:29.972868    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:03:34.106946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:34.106915    5561 generic.go:356] "Generic (PLEG): container finished" podID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerID="cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d" exitCode=0
Oct 30 22:03:34.107333 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:34.106986    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" event={"ID":"fef9544f-6a1c-4289-997c-532d42ba7d38","Type":"ContainerDied","Data":"cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d"}
Oct 30 22:03:34.107468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:34.107452    5561 scope.go:122] "RemoveContainer" containerID="cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d"
Oct 30 22:03:34.109014 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:34.108985    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:34.109082 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:34.109053    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:34.109121 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:34.109085    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:03:35.115897 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:35.115861    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" event={"ID":"fef9544f-6a1c-4289-997c-532d42ba7d38","Type":"ContainerStarted","Data":"44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9"}
Oct 30 22:03:36.070700 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:36.070669    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:03:36.072116 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:36.072083    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:03:36.072340 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:36.072145    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:03:37.070806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:37.070771    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:03:37.071219 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:37.070826    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:03:38.071376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:38.070995    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:03:38.071376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:38.071073    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:03:39.071322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:39.071279    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:03:39.071520 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:39.071379    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:03:39.761379 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:39.758433    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:03:40.070922 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:40.070832    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:03:40.070922 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:40.070895    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:03:41.075260 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:41.075229    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:03:41.076136 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:03:41.076113    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:03:52.982384 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:03:52.982327    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:04:06.645523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.645489    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"]
Oct 30 22:04:06.648567 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.648549    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.745394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.745369    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-99f69c-openshift-default-dockercfg-rhqwf\""
Oct 30 22:04:06.753464 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753436    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-token\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.753599 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753554    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-envoy\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.753647 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753628    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.753719 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753705    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-podinfo\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.753760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753732    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-certs\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.753852 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753831    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28sxp\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-kube-api-access-28sxp\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.753917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753894    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92248b6a-b346-406f-abe3-d3cc80802e1f-istiod-ca-cert\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.753973 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.753960    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-data\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.754062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.754048    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-credential-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.757420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.757395    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"]
Oct 30 22:04:06.855244 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855215    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-envoy\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855273    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855315    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-podinfo\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855334    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-certs\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855426    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28sxp\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-kube-api-access-28sxp\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855695 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855475    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92248b6a-b346-406f-abe3-d3cc80802e1f-istiod-ca-cert\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855695 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855534    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-data\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855695 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855601    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-credential-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.855695 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855669    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-token\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.856084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855768    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.856084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.855827    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-certs\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.856187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.856092    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-data\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.856187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.856122    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-credential-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.856429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.856402    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92248b6a-b346-406f-abe3-d3cc80802e1f-istiod-ca-cert\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.857682 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.857662    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-envoy\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.858165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.858142    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-podinfo\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.915468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.915398    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-token\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.915468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.915419    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28sxp\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-kube-api-access-28sxp\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:06.961283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:06.961259    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:07.387604 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.387574    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"]
Oct 30 22:04:07.387978 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:04:07.387952    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92248b6a_b346_406f_abe3_d3cc80802e1f.slice/crio-17b2abd903d7f4fa698bbbb25d6d4cc1482f0eecd7436476bba29062b3f4577f WatchSource:0}: Error finding container 17b2abd903d7f4fa698bbbb25d6d4cc1482f0eecd7436476bba29062b3f4577f: Status 404 returned error can't find the container with id 17b2abd903d7f4fa698bbbb25d6d4cc1482f0eecd7436476bba29062b3f4577f
Oct 30 22:04:07.390776 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.390740    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:07.390871 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.390824    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:07.390940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.390873    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:07.478043 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.478012    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" event={"ID":"92248b6a-b346-406f-abe3-d3cc80802e1f","Type":"ContainerStarted","Data":"6886a9a49425d81b6cf1965ef946845986cd7dc9a065f7b452e00bfd1ac85a2a"}
Oct 30 22:04:07.478192 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.478076    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" event={"ID":"92248b6a-b346-406f-abe3-d3cc80802e1f","Type":"ContainerStarted","Data":"17b2abd903d7f4fa698bbbb25d6d4cc1482f0eecd7436476bba29062b3f4577f"}
Oct 30 22:04:07.602098 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.602047    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podStartSLOduration=1.602033097 podStartE2EDuration="1.602033097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:07.59819433 +0000 UTC m=+1795.875887802" watchObservedRunningTime="2026-10-30 22:04:07.602033097 +0000 UTC m=+1795.879726568"
Oct 30 22:04:07.961587 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:07.961554    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:08.964083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:08.964042    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:04:08.964760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:08.964723    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:04:09.984376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:09.969807    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:04:09.984376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:09.969867    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:04:10.967374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:10.966804    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:04:10.967374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:10.966865    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:04:11.962400 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:11.962340    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:04:11.971374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:11.965846    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:04:12.229531 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:12.229455    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:04:12.235388 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:12.235335    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:04:12.967736 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:12.967697    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:04:12.968331 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:12.968298    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:04:13.968649 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:13.968606    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:04:13.969070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:13.968676    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:04:14.966214 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:14.966184    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:14.966596 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:14.966573    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:14.967206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:14.967186    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:04:23.000673 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:04:23.000593    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:04:40.903978 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:40.903919    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:04:52.520986 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:52.520948    5561 generic.go:356] "Generic (PLEG): container finished" podID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerID="3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8" exitCode=0
Oct 30 22:04:52.521447 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:52.521018    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" event={"ID":"24ea8511-f145-4af1-95bc-6e10dc90379c","Type":"ContainerDied","Data":"3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8"}
Oct 30 22:04:52.521513 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:52.521466    5561 scope.go:122] "RemoveContainer" containerID="3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8"
Oct 30 22:04:52.523156 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:52.523123    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:52.523244 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:52.523195    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:52.523244 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:52.523224    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:04:52.984229 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:04:52.984191    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:04:53.530017 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:53.529976    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" event={"ID":"24ea8511-f145-4af1-95bc-6e10dc90379c","Type":"ContainerStarted","Data":"4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99"}
Oct 30 22:04:53.714009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:53.712455    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:04:53.714500 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:53.714471    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:04:53.714712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:53.714687    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:04:54.712858 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:54.712819    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.713249 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:54.712872    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:04:55.712616 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:55.712574    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:04:55.712785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:55.712640    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:04:56.712441 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:56.712404    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:04:56.713010 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:56.712975    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:04:56.924872 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:56.924838    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:04:57.712628 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:57.712589    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:04:57.712987 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:57.712647    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:04:58.713465 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:58.713429    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:04:58.713862 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:58.713480    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:04:59.716756 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:59.716724    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:04:59.717834 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:04:59.717816    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:05:12.301785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:12.301755    5561 scope.go:122] "RemoveContainer" containerID="b0dbb1ccce264a35156489a57b61daf1350512a95a8079849d34312d18e22e55"
Oct 30 22:05:21.476002 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:21.475930    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:05:22.982680 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:05:22.982644    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:05:24.311548 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:24.311513    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"]
Oct 30 22:05:24.427544 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:24.427511    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"]
Oct 30 22:05:29.312265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.312224    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh" podUID="d99bcb54-f104-4351-b3ca-8b132b8f662d" containerName="payload-processing" containerID="cri-o://63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85" gracePeriod=25
Oct 30 22:05:29.428404 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.428371    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2" podUID="78de8a1e-3c7c-4a67-9228-355e9083e0d5" containerName="payload-pre-processing" containerID="cri-o://18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d" gracePeriod=25
Oct 30 22:05:29.602530 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.602495    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:05:29.711881 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.711856    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:05:29.767655 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.767620    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d99bcb54-f104-4351-b3ca-8b132b8f662d-kube-api-access-48g9b\" (UniqueName: \"kubernetes.io/projected/d99bcb54-f104-4351-b3ca-8b132b8f662d-kube-api-access-48g9b\") pod \"d99bcb54-f104-4351-b3ca-8b132b8f662d\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") "
Oct 30 22:05:29.767838 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.767719    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d99bcb54-f104-4351-b3ca-8b132b8f662d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99bcb54-f104-4351-b3ca-8b132b8f662d-plugins-config-volume\") pod \"d99bcb54-f104-4351-b3ca-8b132b8f662d\" (UID: \"d99bcb54-f104-4351-b3ca-8b132b8f662d\") "
Oct 30 22:05:29.768511 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.768477    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d99bcb54-f104-4351-b3ca-8b132b8f662d-plugins-config-volume" pod "d99bcb54-f104-4351-b3ca-8b132b8f662d" (UID: "d99bcb54-f104-4351-b3ca-8b132b8f662d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:29.768731 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.768709    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99bcb54-f104-4351-b3ca-8b132b8f662d-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:29.770108 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.770078    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99bcb54-f104-4351-b3ca-8b132b8f662d-kube-api-access-48g9b" pod "d99bcb54-f104-4351-b3ca-8b132b8f662d" (UID: "d99bcb54-f104-4351-b3ca-8b132b8f662d"). InnerVolumeSpecName "kube-api-access-48g9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:29.869673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.869639    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78de8a1e-3c7c-4a67-9228-355e9083e0d5-kube-api-access-fzqh4\" (UniqueName: \"kubernetes.io/projected/78de8a1e-3c7c-4a67-9228-355e9083e0d5-kube-api-access-fzqh4\") pod \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") "
Oct 30 22:05:29.869821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.869768    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78de8a1e-3c7c-4a67-9228-355e9083e0d5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78de8a1e-3c7c-4a67-9228-355e9083e0d5-plugins-config-volume\") pod \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\" (UID: \"78de8a1e-3c7c-4a67-9228-355e9083e0d5\") "
Oct 30 22:05:29.870145 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.870112    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78de8a1e-3c7c-4a67-9228-355e9083e0d5-plugins-config-volume" pod "78de8a1e-3c7c-4a67-9228-355e9083e0d5" (UID: "78de8a1e-3c7c-4a67-9228-355e9083e0d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:29.870504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.870478    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78de8a1e-3c7c-4a67-9228-355e9083e0d5-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:29.870610 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.870509    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48g9b\" (UniqueName: \"kubernetes.io/projected/d99bcb54-f104-4351-b3ca-8b132b8f662d-kube-api-access-48g9b\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:29.871903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.871881    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78de8a1e-3c7c-4a67-9228-355e9083e0d5-kube-api-access-fzqh4" pod "78de8a1e-3c7c-4a67-9228-355e9083e0d5" (UID: "78de8a1e-3c7c-4a67-9228-355e9083e0d5"). InnerVolumeSpecName "kube-api-access-fzqh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:29.971780 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:29.971746    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzqh4\" (UniqueName: \"kubernetes.io/projected/78de8a1e-3c7c-4a67-9228-355e9083e0d5-kube-api-access-fzqh4\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:30.051401 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.051336    5561 generic.go:356] "Generic (PLEG): container finished" podID="78de8a1e-3c7c-4a67-9228-355e9083e0d5" containerID="18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d" exitCode=0
Oct 30 22:05:30.051564 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.051427    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"
Oct 30 22:05:30.051564 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.051434    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2" event={"ID":"78de8a1e-3c7c-4a67-9228-355e9083e0d5","Type":"ContainerDied","Data":"18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d"}
Oct 30 22:05:30.051564 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.051495    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2" event={"ID":"78de8a1e-3c7c-4a67-9228-355e9083e0d5","Type":"ContainerDied","Data":"a563fceafa959eb9df0622d06457ebbfd7376f6054713891bb2a73af15da231d"}
Oct 30 22:05:30.051564 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.051524    5561 scope.go:122] "RemoveContainer" containerID="18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d"
Oct 30 22:05:30.053672 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.053632    5561 generic.go:356] "Generic (PLEG): container finished" podID="d99bcb54-f104-4351-b3ca-8b132b8f662d" containerID="63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85" exitCode=0
Oct 30 22:05:30.053795 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.053672    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh" event={"ID":"d99bcb54-f104-4351-b3ca-8b132b8f662d","Type":"ContainerDied","Data":"63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85"}
Oct 30 22:05:30.053795 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.053697    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"
Oct 30 22:05:30.053795 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.053715    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh" event={"ID":"d99bcb54-f104-4351-b3ca-8b132b8f662d","Type":"ContainerDied","Data":"734f90c7990e9a032578514ac9fd97fb085428cd37df9810aed77e5df8091071"}
Oct 30 22:05:30.063569 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.063467    5561 scope.go:122] "RemoveContainer" containerID="18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d"
Oct 30 22:05:30.063747 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:05:30.063727    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d\": container with ID starting with 18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d not found: ID does not exist" containerID="18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d"
Oct 30 22:05:30.063805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.063754    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d"} err="failed to get container status \"18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d\": rpc error: code = NotFound desc = could not find container \"18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d\": container with ID starting with 18908409a58b98bdacb838b50be7cf2fc6808aa604490159bfedf9cf956ca30d not found: ID does not exist"
Oct 30 22:05:30.063805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.063769    5561 scope.go:122] "RemoveContainer" containerID="63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85"
Oct 30 22:05:30.073307 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.073285    5561 scope.go:122] "RemoveContainer" containerID="63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85"
Oct 30 22:05:30.073645 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:05:30.073626    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85\": container with ID starting with 63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85 not found: ID does not exist" containerID="63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85"
Oct 30 22:05:30.073722 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.073654    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85"} err="failed to get container status \"63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85\": rpc error: code = NotFound desc = could not find container \"63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85\": container with ID starting with 63875c00c2c974aa56adfe3612d67972ae9eceec92106eb2b0c909607033fd85 not found: ID does not exist"
Oct 30 22:05:30.221152 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.221123    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"]
Oct 30 22:05:30.270378 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.270323    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh"]
Oct 30 22:05:30.333590 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.333559    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"]
Oct 30 22:05:30.339408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:30.339386    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2"]
Oct 30 22:05:31.847638 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:31.847607    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78de8a1e-3c7c-4a67-9228-355e9083e0d5" path="/var/lib/kubelet/pods/78de8a1e-3c7c-4a67-9228-355e9083e0d5/volumes"
Oct 30 22:05:31.848121 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:31.848099    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d99bcb54-f104-4351-b3ca-8b132b8f662d" path="/var/lib/kubelet/pods/d99bcb54-f104-4351-b3ca-8b132b8f662d/volumes"
Oct 30 22:05:46.939245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:46.939211    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"]
Oct 30 22:05:46.939797 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:46.939439    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" containerID="cri-o://c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b" gracePeriod=30
Oct 30 22:05:49.348800 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:49.348765    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"]
Oct 30 22:05:49.349227 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:49.348969    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy" containerID="cri-o://c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe" gracePeriod=30
Oct 30 22:05:50.414766 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:50.414735    5561 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:05:50.415142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:50.414788    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:05:52.298551 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.298524    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 22:05:52.348105 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348071    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-certs\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348235 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348179    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-credential-socket\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348235 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348199    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37a75d0b-977e-4970-81b9-b6665bbeb6db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a75d0b-977e-4970-81b9-b6665bbeb6db-istiod-ca-cert\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348235 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348222    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-token\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-token\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348247    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-data\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348441 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348338    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-certs" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:52.348491 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348434    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-socket\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348491 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348478    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-kube-api-access-8fsvc\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-kube-api-access-8fsvc\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348501    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-envoy\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348532    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-credential-socket" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:52.348600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348587    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-podinfo\") pod \"37a75d0b-977e-4970-81b9-b6665bbeb6db\" (UID: \"37a75d0b-977e-4970-81b9-b6665bbeb6db\") "
Oct 30 22:05:52.348764 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348683    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37a75d0b-977e-4970-81b9-b6665bbeb6db-istiod-ca-cert" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:52.348764 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.348688    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-data" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:52.349039 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.349018    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-socket" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:52.349230 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.349214    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.349303 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.349236    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.349303 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.349251    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.349303 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.349265    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a75d0b-977e-4970-81b9-b6665bbeb6db-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.349303 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.349278    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.350741 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.350712    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-podinfo" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:52.350741 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.350725    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-envoy" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:52.350881 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.350852    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-token" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:52.350972 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.350957    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-kube-api-access-8fsvc" pod "37a75d0b-977e-4970-81b9-b6665bbeb6db" (UID: "37a75d0b-977e-4970-81b9-b6665bbeb6db"). InnerVolumeSpecName "kube-api-access-8fsvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:52.382861 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.382827    5561 generic.go:356] "Generic (PLEG): container finished" podID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerID="c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b" exitCode=0
Oct 30 22:05:52.382976 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.382880    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"
Oct 30 22:05:52.382976 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.382912    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" event={"ID":"37a75d0b-977e-4970-81b9-b6665bbeb6db","Type":"ContainerDied","Data":"c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b"}
Oct 30 22:05:52.383103 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.382983    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94" event={"ID":"37a75d0b-977e-4970-81b9-b6665bbeb6db","Type":"ContainerDied","Data":"71fc0f95f9d43c11abcf1fa585f28b21e6e9b751153131d3358905295a4846d0"}
Oct 30 22:05:52.383103 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.383016    5561 scope.go:122] "RemoveContainer" containerID="c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b"
Oct 30 22:05:52.393878 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.393858    5561 scope.go:122] "RemoveContainer" containerID="c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b"
Oct 30 22:05:52.394194 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:05:52.394174    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b\": container with ID starting with c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b not found: ID does not exist" containerID="c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b"
Oct 30 22:05:52.394274 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.394205    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b"} err="failed to get container status \"c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b\": rpc error: code = NotFound desc = could not find container \"c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b\": container with ID starting with c51abf5ed4fe805b7c72fdf75336a87768a72640316e364f52cb2ca761dd7f8b not found: ID does not exist"
Oct 30 22:05:52.450245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.450219    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.450245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.450243    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fsvc\" (UniqueName: \"kubernetes.io/projected/37a75d0b-977e-4970-81b9-b6665bbeb6db-kube-api-access-8fsvc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.450438 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.450255    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.450438 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.450264    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a75d0b-977e-4970-81b9-b6665bbeb6db-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:52.669374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.661216    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"]
Oct 30 22:05:52.669374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:52.666872    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94"]
Oct 30 22:05:53.007382 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:05:53.007286    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:05:53.847611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:53.847570    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" path="/var/lib/kubelet/pods/37a75d0b-977e-4970-81b9-b6665bbeb6db/volumes"
Oct 30 22:05:54.618100 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.618079    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 22:05:54.702490 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702466    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-envoy\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.702642 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702526    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-data\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.702642 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702554    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0ff8aad-679b-4686-9b94-fb4441696b67-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ff8aad-679b-4686-9b94-fb4441696b67-istiod-ca-cert\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.702642 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702598    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-credential-socket\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.702642 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702632    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-certs\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.702863 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702748    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-kube-api-access-xrstj\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-kube-api-access-xrstj\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.702863 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702829    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-credential-socket" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:54.702863 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702846    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-data" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:54.703009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702877    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-socket\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.703009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702922    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-token\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-token\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.703009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.702956    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-podinfo\") pod \"a0ff8aad-679b-4686-9b94-fb4441696b67\" (UID: \"a0ff8aad-679b-4686-9b94-fb4441696b67\") "
Oct 30 22:05:54.703168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703008    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-certs" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:54.703168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703008    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ff8aad-679b-4686-9b94-fb4441696b67-istiod-ca-cert" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:54.703168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703124    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-socket" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:54.703652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703627    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.703775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703658    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.703775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703674    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.703775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703688    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ff8aad-679b-4686-9b94-fb4441696b67-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.703775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.703705    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.704692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.704666    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-envoy" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:54.705070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.705045    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-kube-api-access-xrstj" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "kube-api-access-xrstj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:54.705070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.705055    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-podinfo" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:54.705187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.705069    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-token" pod "a0ff8aad-679b-4686-9b94-fb4441696b67" (UID: "a0ff8aad-679b-4686-9b94-fb4441696b67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:54.805263 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.805231    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrstj\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-kube-api-access-xrstj\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.805263 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.805261    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.805263 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.805271    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:54.805675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:54.805280    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ff8aad-679b-4686-9b94-fb4441696b67-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.369561    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"]
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370486    5561 cpu_manager.go:414] "containerMap: removing container" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370506    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370537    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370544    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370564    5561 cpu_manager.go:414] "containerMap: removing container" podUID="78de8a1e-3c7c-4a67-9228-355e9083e0d5" containerName="payload-pre-processing"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370571    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78de8a1e-3c7c-4a67-9228-355e9083e0d5" containerName="payload-pre-processing"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370589    5561 cpu_manager.go:414] "containerMap: removing container" podUID="d99bcb54-f104-4351-b3ca-8b132b8f662d" containerName="payload-processing"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370596    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d99bcb54-f104-4351-b3ca-8b132b8f662d" containerName="payload-processing"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370778    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="37a75d0b-977e-4970-81b9-b6665bbeb6db" containerName="istio-proxy"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370796    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerName="istio-proxy"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370809    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="78de8a1e-3c7c-4a67-9228-355e9083e0d5" containerName="payload-pre-processing"
Oct 30 22:05:55.373428 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.370820    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="d99bcb54-f104-4351-b3ca-8b132b8f662d" containerName="payload-processing"
Oct 30 22:05:55.415401 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.403599    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.415401 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.415009    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"]
Oct 30 22:05:55.458380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.457395    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fppg\" (UniqueName: \"kubernetes.io/projected/3788186f-ba6c-447e-babd-a036714b5b17-kube-api-access-9fppg\") pod \"payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.458380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.457526    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3788186f-ba6c-447e-babd-a036714b5b17-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.461480 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.460262    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w6-1db768\""
Oct 30 22:05:55.461480 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.460464    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w6-1db768-dockercfg-dzh4d\""
Oct 30 22:05:55.501714 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.501672    5561 generic.go:356] "Generic (PLEG): container finished" podID="a0ff8aad-679b-4686-9b94-fb4441696b67" containerID="c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe" exitCode=0
Oct 30 22:05:55.553712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.504001    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.504195    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" event={"ID":"a0ff8aad-679b-4686-9b94-fb4441696b67","Type":"ContainerDied","Data":"c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe"}
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.595334    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"]
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.560875    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fppg\" (UniqueName: \"kubernetes.io/projected/3788186f-ba6c-447e-babd-a036714b5b17-kube-api-access-9fppg\") pod \"payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.598510    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3788186f-ba6c-447e-babd-a036714b5b17-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.614713    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3788186f-ba6c-447e-babd-a036714b5b17-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.614856    5561 scope.go:122] "RemoveContainer" containerID="c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe"
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.649885    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj" event={"ID":"a0ff8aad-679b-4686-9b94-fb4441696b67","Type":"ContainerDied","Data":"e8d9e0aa76b90dd2f43ba9d7f631b5853c5b5f0f46aa4399107d48024a310d49"}
Oct 30 22:05:55.688048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.650080    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:55.738587 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.738545    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fppg\" (UniqueName: \"kubernetes.io/projected/3788186f-ba6c-447e-babd-a036714b5b17-kube-api-access-9fppg\") pod \"payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.752986 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.751266    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4czfq\" (UniqueName: \"kubernetes.io/projected/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-kube-api-access-4czfq\") pod \"payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:55.752986 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.751386    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:55.767113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.757091    5561 scope.go:122] "RemoveContainer" containerID="c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe"
Oct 30 22:05:55.767113 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:05:55.763043    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe\": container with ID starting with c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe not found: ID does not exist" containerID="c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe"
Oct 30 22:05:55.767113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.763084    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe"} err="failed to get container status \"c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe\": rpc error: code = NotFound desc = could not find container \"c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe\": container with ID starting with c8eb0744f2c8f5b57ca9a94d8ed098e9d1b691a6ce70817fa29c8c900c0e3fbe not found: ID does not exist"
Oct 30 22:05:55.769481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.769456    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"]
Oct 30 22:05:55.784562 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.784541    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:55.857225 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.853859    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4czfq\" (UniqueName: \"kubernetes.io/projected/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-kube-api-access-4czfq\") pod \"payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:55.857225 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.853937    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:55.857225 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.855615    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:55.912259 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.912227    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4czfq\" (UniqueName: \"kubernetes.io/projected/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-kube-api-access-4czfq\") pod \"payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:55.979888 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.979820    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"]
Oct 30 22:05:55.986214 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:55.986185    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj"]
Oct 30 22:05:56.076034 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.076003    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:56.166115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.166082    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"]
Oct 30 22:05:56.458025 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:05:56.457991    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3c27d6_7a97_4350_9e5f_a5d2c7aaafca.slice/crio-503b7d204becfa389fccb4ebe3843613d79a8d81f6b5dc6bfe4ee292ff958a3a WatchSource:0}: Error finding container 503b7d204becfa389fccb4ebe3843613d79a8d81f6b5dc6bfe4ee292ff958a3a: Status 404 returned error can't find the container with id 503b7d204becfa389fccb4ebe3843613d79a8d81f6b5dc6bfe4ee292ff958a3a
Oct 30 22:05:56.458406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.458044    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"]
Oct 30 22:05:56.562554 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.562522    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p" event={"ID":"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca","Type":"ContainerStarted","Data":"5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4"}
Oct 30 22:05:56.562675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.562576    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p" event={"ID":"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca","Type":"ContainerStarted","Data":"503b7d204becfa389fccb4ebe3843613d79a8d81f6b5dc6bfe4ee292ff958a3a"}
Oct 30 22:05:56.562675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.562626    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:05:56.565830 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.565799    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8" event={"ID":"3788186f-ba6c-447e-babd-a036714b5b17","Type":"ContainerStarted","Data":"63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5"}
Oct 30 22:05:56.565955 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.565853    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8" event={"ID":"3788186f-ba6c-447e-babd-a036714b5b17","Type":"ContainerStarted","Data":"98b98c8768eac5c0aad460d8bf612d773b30b15b68a2788bf81c33be33aae5f2"}
Oct 30 22:05:56.565955 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.565881    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:05:56.683168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.683120    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p" podStartSLOduration=1.683107828 podStartE2EDuration="1.683107828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:56.680862117 +0000 UTC m=+1904.958555612" watchObservedRunningTime="2026-10-30 22:05:56.683107828 +0000 UTC m=+1904.960801298"
Oct 30 22:05:56.819425 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:56.805225    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8" podStartSLOduration=1.8052037570000001 podStartE2EDuration="1.805203757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:56.803785883 +0000 UTC m=+1905.081479355" watchObservedRunningTime="2026-10-30 22:05:56.805203757 +0000 UTC m=+1905.082897229"
Oct 30 22:05:57.847431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:05:57.847397    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0ff8aad-679b-4686-9b94-fb4441696b67" path="/var/lib/kubelet/pods/a0ff8aad-679b-4686-9b94-fb4441696b67/volumes"
Oct 30 22:06:05.459374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.440446    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"]
Oct 30 22:06:05.526617 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.526580    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"]
Oct 30 22:06:05.528027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.527938    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.657373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.657049    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7flkk\" (UniqueName: \"kubernetes.io/projected/66a9c077-71c3-468b-b5fd-12d6ba862263-kube-api-access-7flkk\") pod \"payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.657373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.657309    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a9c077-71c3-468b-b5fd-12d6ba862263-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.785547 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.756498    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-6cd200-dockercfg-lmj29\""
Oct 30 22:06:05.785547 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.757402    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-6cd200\""
Oct 30 22:06:05.785547 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.776788    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a9c077-71c3-468b-b5fd-12d6ba862263-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.785547 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.776938    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7flkk\" (UniqueName: \"kubernetes.io/projected/66a9c077-71c3-468b-b5fd-12d6ba862263-kube-api-access-7flkk\") pod \"payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.811571 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.811532    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a9c077-71c3-468b-b5fd-12d6ba862263-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.831379 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.827447    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"]
Oct 30 22:06:05.865906 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.865783    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7flkk\" (UniqueName: \"kubernetes.io/projected/66a9c077-71c3-468b-b5fd-12d6ba862263-kube-api-access-7flkk\") pod \"payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.867406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.867369    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:05.878293 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.878264    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xncxg\" (UniqueName: \"kubernetes.io/projected/9195b46b-8260-4b7a-a879-20c060475ed6-kube-api-access-xncxg\") pod \"payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:05.878574 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.878546    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9195b46b-8260-4b7a-a879-20c060475ed6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:05.878691 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.878589    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"]
Oct 30 22:06:05.886565 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.886536    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:05.979646 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.979615    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xncxg\" (UniqueName: \"kubernetes.io/projected/9195b46b-8260-4b7a-a879-20c060475ed6-kube-api-access-xncxg\") pod \"payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:05.979795 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.979783    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9195b46b-8260-4b7a-a879-20c060475ed6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:05.981131 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:05.981098    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9195b46b-8260-4b7a-a879-20c060475ed6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:06.037408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.037323    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xncxg\" (UniqueName: \"kubernetes.io/projected/9195b46b-8260-4b7a-a879-20c060475ed6-kube-api-access-xncxg\") pod \"payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:06.194271 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.194239    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:06.262392 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.261706    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"]
Oct 30 22:06:06.264851 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:06:06.264823    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a9c077_71c3_468b_b5fd_12d6ba862263.slice/crio-e3362612a62c2426e2b735ac559a7c604a432328c5ac792d99b4e5a2abf23b0f WatchSource:0}: Error finding container e3362612a62c2426e2b735ac559a7c604a432328c5ac792d99b4e5a2abf23b0f: Status 404 returned error can't find the container with id e3362612a62c2426e2b735ac559a7c604a432328c5ac792d99b4e5a2abf23b0f
Oct 30 22:06:06.573443 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.573409    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"]
Oct 30 22:06:06.574258 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:06:06.574233    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9195b46b_8260_4b7a_a879_20c060475ed6.slice/crio-63cc0708f787281a8827e7b4948b6872b91f80e72dbe535c347a804797933e11 WatchSource:0}: Error finding container 63cc0708f787281a8827e7b4948b6872b91f80e72dbe535c347a804797933e11: Status 404 returned error can't find the container with id 63cc0708f787281a8827e7b4948b6872b91f80e72dbe535c347a804797933e11
Oct 30 22:06:06.865927 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.865852    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5" event={"ID":"9195b46b-8260-4b7a-a879-20c060475ed6","Type":"ContainerStarted","Data":"f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931"}
Oct 30 22:06:06.865927 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.865906    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5" event={"ID":"9195b46b-8260-4b7a-a879-20c060475ed6","Type":"ContainerStarted","Data":"63cc0708f787281a8827e7b4948b6872b91f80e72dbe535c347a804797933e11"}
Oct 30 22:06:06.866158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.865937    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:06.867555 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.867533    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm" event={"ID":"66a9c077-71c3-468b-b5fd-12d6ba862263","Type":"ContainerStarted","Data":"6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d"}
Oct 30 22:06:06.867660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.867576    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm" event={"ID":"66a9c077-71c3-468b-b5fd-12d6ba862263","Type":"ContainerStarted","Data":"e3362612a62c2426e2b735ac559a7c604a432328c5ac792d99b4e5a2abf23b0f"}
Oct 30 22:06:06.867660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.867616    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:06.984909 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:06.984839    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5" podStartSLOduration=1.984803944 podStartE2EDuration="1.984803944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:06.980602464 +0000 UTC m=+1915.258295938" watchObservedRunningTime="2026-10-30 22:06:06.984803944 +0000 UTC m=+1915.262497416"
Oct 30 22:06:07.142860 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:07.142394    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm" podStartSLOduration=2.142331743 podStartE2EDuration="2.142331743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:07.120717235 +0000 UTC m=+1915.398410707" watchObservedRunningTime="2026-10-30 22:06:07.142331743 +0000 UTC m=+1915.420025215"
Oct 30 22:06:07.575420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:07.575306    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:06:07.575767 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:07.575498    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:06:17.877307 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:17.877277    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:06:17.877891 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:17.877338    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:06:23.047079 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:06:23.047044    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:06:40.793735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:40.793700    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"]
Oct 30 22:06:40.798152 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:40.798132    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:40.854856 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:40.854821    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"]
Oct 30 22:06:41.202377 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.136219    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-9c8c4d-dockercfg-5rjzv\""
Oct 30 22:06:41.202377 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.136479    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-9c8c4d\""
Oct 30 22:06:41.228320 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.228266    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:41.229013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.228981    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv9xz\" (UniqueName: \"kubernetes.io/projected/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-kube-api-access-tv9xz\") pod \"payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:41.332468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.328784    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"]
Oct 30 22:06:41.374408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.340442    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv9xz\" (UniqueName: \"kubernetes.io/projected/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-kube-api-access-tv9xz\") pod \"payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:41.374408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.353235    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:41.414377 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.414092    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:41.424399 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.422099    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"]
Oct 30 22:06:41.424399 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.423844    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:41.425539 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.425444    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:06:41.453209 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.452895    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv9xz\" (UniqueName: \"kubernetes.io/projected/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-kube-api-access-tv9xz\") pod \"payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:41.525409 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.525335    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fcbc277-8220-4a5e-865e-1bba7d499d29-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:41.525587 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.525507    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkz5n\" (UniqueName: \"kubernetes.io/projected/6fcbc277-8220-4a5e-865e-1bba7d499d29-kube-api-access-dkz5n\") pod \"payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:41.627414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.627384    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fcbc277-8220-4a5e-865e-1bba7d499d29-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:41.627579 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.627476    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkz5n\" (UniqueName: \"kubernetes.io/projected/6fcbc277-8220-4a5e-865e-1bba7d499d29-kube-api-access-dkz5n\") pod \"payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:41.628085 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.628060    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fcbc277-8220-4a5e-865e-1bba7d499d29-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:41.685924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.685895    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkz5n\" (UniqueName: \"kubernetes.io/projected/6fcbc277-8220-4a5e-865e-1bba7d499d29-kube-api-access-dkz5n\") pod \"payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:41.712069 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.712018    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:41.751424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:41.751396    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:42.098278 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.098199    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"]
Oct 30 22:06:42.136843 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:06:42.136814    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f06cd4_015e_4e94_aaaf_0bde5c12d321.slice/crio-9f28ab0a245a0ce47cb397856f6459edf275016c0574155d4c83b5e149819343 WatchSource:0}: Error finding container 9f28ab0a245a0ce47cb397856f6459edf275016c0574155d4c83b5e149819343: Status 404 returned error can't find the container with id 9f28ab0a245a0ce47cb397856f6459edf275016c0574155d4c83b5e149819343
Oct 30 22:06:42.148634 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.148610    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"]
Oct 30 22:06:42.150112 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:06:42.150083    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fcbc277_8220_4a5e_865e_1bba7d499d29.slice/crio-fe5579586845c5f40e0ecafbaef98582d8a764167702dd599050a3a245ca292a WatchSource:0}: Error finding container fe5579586845c5f40e0ecafbaef98582d8a764167702dd599050a3a245ca292a: Status 404 returned error can't find the container with id fe5579586845c5f40e0ecafbaef98582d8a764167702dd599050a3a245ca292a
Oct 30 22:06:42.394942 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.393637    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd" event={"ID":"6fcbc277-8220-4a5e-865e-1bba7d499d29","Type":"ContainerStarted","Data":"3e443ba3e855cfb0fb57e0464888eba24b088e4f8effbd063e1766a4b4dcabce"}
Oct 30 22:06:42.394942 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.393708    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd" event={"ID":"6fcbc277-8220-4a5e-865e-1bba7d499d29","Type":"ContainerStarted","Data":"fe5579586845c5f40e0ecafbaef98582d8a764167702dd599050a3a245ca292a"}
Oct 30 22:06:42.394942 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.394883    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:42.413852 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.413806    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n" event={"ID":"b6f06cd4-015e-4e94-aaaf-0bde5c12d321","Type":"ContainerStarted","Data":"fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6"}
Oct 30 22:06:42.414133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.414105    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n" event={"ID":"b6f06cd4-015e-4e94-aaaf-0bde5c12d321","Type":"ContainerStarted","Data":"9f28ab0a245a0ce47cb397856f6459edf275016c0574155d4c83b5e149819343"}
Oct 30 22:06:42.415223 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.415201    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:42.534009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:42.531968    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd" podStartSLOduration=2.531948098 podStartE2EDuration="2.531948098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:42.517941842 +0000 UTC m=+1950.795635305" watchObservedRunningTime="2026-10-30 22:06:42.531948098 +0000 UTC m=+1950.809641571"
Oct 30 22:06:52.982795 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:06:52.982756    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:06:54.443997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:54.443967    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:06:54.444425 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:54.444199    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:06:54.561856 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:06:54.561805    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n" podStartSLOduration=14.561791203 podStartE2EDuration="14.561791203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:42.671555572 +0000 UTC m=+1950.949249053" watchObservedRunningTime="2026-10-30 22:06:54.561791203 +0000 UTC m=+1962.839484673"
Oct 30 22:07:03.950832 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:03.950798    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"]
Oct 30 22:07:03.975371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:03.956939    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"]
Oct 30 22:07:03.975706 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:03.965165    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.029600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.029574    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-78cf83\""
Oct 30 22:07:04.080016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.079983    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-78cf83-dockercfg-dxqr6\""
Oct 30 22:07:04.118059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.118027    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") " pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.118320 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.118298    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zvtm\" (UniqueName: \"kubernetes.io/projected/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-kube-api-access-4zvtm\") pod \"payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") " pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.219850 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.219780    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zvtm\" (UniqueName: \"kubernetes.io/projected/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-kube-api-access-4zvtm\") pod \"payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") " pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.220000 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.219873    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") " pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.220776 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.220752    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") " pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.278470 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.278436    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zvtm\" (UniqueName: \"kubernetes.io/projected/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-kube-api-access-4zvtm\") pod \"payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") " pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.304414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.304389    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:04.714818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:04.712762    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"]
Oct 30 22:07:04.718517 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:07:04.718385    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce6b173_dc1b_4f8a_817a_0299b3e8fbab.slice/crio-b6fa8874fc6bde30052f9060bf9f41ca88b2215dcff9bd4b4eb33c67f8dff929 WatchSource:0}: Error finding container b6fa8874fc6bde30052f9060bf9f41ca88b2215dcff9bd4b4eb33c67f8dff929: Status 404 returned error can't find the container with id b6fa8874fc6bde30052f9060bf9f41ca88b2215dcff9bd4b4eb33c67f8dff929
Oct 30 22:07:05.724262 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:05.724214    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4" event={"ID":"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab","Type":"ContainerStarted","Data":"21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1"}
Oct 30 22:07:05.724698 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:05.724471    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:05.724698 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:05.724505    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4" event={"ID":"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab","Type":"ContainerStarted","Data":"b6fa8874fc6bde30052f9060bf9f41ca88b2215dcff9bd4b4eb33c67f8dff929"}
Oct 30 22:07:05.844376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:05.844308    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4" podStartSLOduration=2.844293984 podStartE2EDuration="2.844293984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:05.841237253 +0000 UTC m=+1974.118930723" watchObservedRunningTime="2026-10-30 22:07:05.844293984 +0000 UTC m=+1974.121987454"
Oct 30 22:07:16.735186 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:16.735154    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:07:22.983126 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:07:22.983053    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:07:47.650373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:07:47.648260    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:07:52.982426 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:07:52.982341    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:08:05.471393 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:05.471343    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"]
Oct 30 22:08:05.471906 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:05.471614    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy" containerID="cri-o://38ca2bca61288b5d49ab8b49136fa58d4e66a5cae1becb63a05fb9e57ff33028" gracePeriod=30
Oct 30 22:08:05.524455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:05.524420    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"]
Oct 30 22:08:05.524720 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:05.524673    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" containerID="cri-o://be2de714aedc492fa564c8673274857383f7f99c2ddbeb0c1474910757f8c797" gracePeriod=30
Oct 30 22:08:06.156365 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:06.156320    5561 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:08:06.156536 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:06.156391    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:10.724003 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.723964    5561 generic.go:356] "Generic (PLEG): container finished" podID="84239b3b-2157-4921-aacd-ed21bc708701" containerID="38ca2bca61288b5d49ab8b49136fa58d4e66a5cae1becb63a05fb9e57ff33028" exitCode=0
Oct 30 22:08:10.724502 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.724124    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" event={"ID":"84239b3b-2157-4921-aacd-ed21bc708701","Type":"ContainerDied","Data":"38ca2bca61288b5d49ab8b49136fa58d4e66a5cae1becb63a05fb9e57ff33028"}
Oct 30 22:08:10.726015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.725994    5561 generic.go:356] "Generic (PLEG): container finished" podID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerID="be2de714aedc492fa564c8673274857383f7f99c2ddbeb0c1474910757f8c797" exitCode=0
Oct 30 22:08:10.726116 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.726027    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" event={"ID":"3d09b71b-828c-4956-9d0b-d2da25d89b50","Type":"ContainerDied","Data":"be2de714aedc492fa564c8673274857383f7f99c2ddbeb0c1474910757f8c797"}
Oct 30 22:08:10.818606 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.818581    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 22:08:10.846696 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.846671    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 22:08:10.940402 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940338    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-envoy\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.940571 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940420    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-kube-api-access-pg7sl\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-kube-api-access-pg7sl\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.940571 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940470    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84239b3b-2157-4921-aacd-ed21bc708701-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84239b3b-2157-4921-aacd-ed21bc708701-istiod-ca-cert\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.940571 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940507    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-data\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.940748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940571    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-credential-socket\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.940748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940596    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-token\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-token\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.940748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940650    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-istio-token\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-istio-token\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.940748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940702    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-certs\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.940957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940798    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-socket\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.940957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940780    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-data" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.940957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940825    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-credential-socket" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.940957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940836    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/84239b3b-2157-4921-aacd-ed21bc708701-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84239b3b-2157-4921-aacd-ed21bc708701-istio-podinfo\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.940957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940921    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84239b3b-2157-4921-aacd-ed21bc708701-istiod-ca-cert" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:10.940957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.940953    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-envoy\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.941291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941063    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-certs\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.941291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941065    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-certs" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.941291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941115    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-podinfo\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.941291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941143    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-credential-socket\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.941291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941202    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-kube-api-access-t2xsl\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-kube-api-access-t2xsl\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.941291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941247    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d09b71b-828c-4956-9d0b-d2da25d89b50-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d09b71b-828c-4956-9d0b-d2da25d89b50-istiod-ca-cert\") pod \"3d09b71b-828c-4956-9d0b-d2da25d89b50\" (UID: \"3d09b71b-828c-4956-9d0b-d2da25d89b50\") "
Oct 30 22:08:10.941665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941382    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-socket\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.941665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941434    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-data\") pod \"84239b3b-2157-4921-aacd-ed21bc708701\" (UID: \"84239b3b-2157-4921-aacd-ed21bc708701\") "
Oct 30 22:08:10.942147 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941207    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-socket" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.942147 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.941553    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-certs" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.942311 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.942185    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-credential-socket" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.942486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.942437    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-socket" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.942611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.942529    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d09b71b-828c-4956-9d0b-d2da25d89b50-istiod-ca-cert" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:10.942892 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.942865    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-data" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.943501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943282    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84239b3b-2157-4921-aacd-ed21bc708701-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943307    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943569    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943594    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943611    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943627    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943642    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943655    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d09b71b-828c-4956-9d0b-d2da25d89b50-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943669    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.943814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.943683    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:10.944712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.944682    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84239b3b-2157-4921-aacd-ed21bc708701-istio-podinfo" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:10.944712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.944695    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-podinfo" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:10.944875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.944849    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-kube-api-access-pg7sl" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "kube-api-access-pg7sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:10.945005 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.944984    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-envoy" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:10.945079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.945009    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-kube-api-access-t2xsl" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "kube-api-access-t2xsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:10.945079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.945029    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-istio-token" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:10.945182 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.945165    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-token" pod "3d09b71b-828c-4956-9d0b-d2da25d89b50" (UID: "3d09b71b-828c-4956-9d0b-d2da25d89b50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:10.945321 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:10.945304    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-envoy" pod "84239b3b-2157-4921-aacd-ed21bc708701" (UID: "84239b3b-2157-4921-aacd-ed21bc708701"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:11.045504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045418    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84239b3b-2157-4921-aacd-ed21bc708701-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.045504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045457    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.045504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045469    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.045504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045483    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2xsl\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-kube-api-access-t2xsl\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.045504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045496    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84239b3b-2157-4921-aacd-ed21bc708701-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.045504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045511    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg7sl\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-kube-api-access-pg7sl\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.045858 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045523    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d09b71b-828c-4956-9d0b-d2da25d89b50-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.045858 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.045535    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84239b3b-2157-4921-aacd-ed21bc708701-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:11.734488 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.734453    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"
Oct 30 22:08:11.734946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.734471    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92" event={"ID":"84239b3b-2157-4921-aacd-ed21bc708701","Type":"ContainerDied","Data":"3d706d1f5ef1eac70f37d87bef26720fdd0659282d731e585dc0e6e554daf57d"}
Oct 30 22:08:11.734946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.734584    5561 scope.go:122] "RemoveContainer" containerID="38ca2bca61288b5d49ab8b49136fa58d4e66a5cae1becb63a05fb9e57ff33028"
Oct 30 22:08:11.736688 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.736656    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k" event={"ID":"3d09b71b-828c-4956-9d0b-d2da25d89b50","Type":"ContainerDied","Data":"f3d2f361600cfde12937cb16e0db0b13097b7a278c0ab18af5ec05b777aed376"}
Oct 30 22:08:11.736813 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.736705    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"
Oct 30 22:08:11.747734 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.747715    5561 scope.go:122] "RemoveContainer" containerID="be2de714aedc492fa564c8673274857383f7f99c2ddbeb0c1474910757f8c797"
Oct 30 22:08:11.903839 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.903809    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"]
Oct 30 22:08:11.954511 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:11.954483    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k"]
Oct 30 22:08:12.122971 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:12.104250    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"]
Oct 30 22:08:12.122971 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:12.120871    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92"]
Oct 30 22:08:13.846974 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:13.846945    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" path="/var/lib/kubelet/pods/3d09b71b-828c-4956-9d0b-d2da25d89b50/volumes"
Oct 30 22:08:13.847466 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:13.847450    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84239b3b-2157-4921-aacd-ed21bc708701" path="/var/lib/kubelet/pods/84239b3b-2157-4921-aacd-ed21bc708701/volumes"
Oct 30 22:08:22.984304 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:08:22.984266    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:08:30.401576 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:30.401542    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:08:35.645797 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:35.645764    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"]
Oct 30 22:08:35.646255 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:35.646045    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" containerID="cri-o://d7ab9b793531c68951239f77192513aab255871e34a33bc6dc5b6ac925b4908f" gracePeriod=30
Oct 30 22:08:38.673798 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:38.673756    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:08:38.674190 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:38.673824    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:41.026197 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.026160    5561 generic.go:356] "Generic (PLEG): container finished" podID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerID="d7ab9b793531c68951239f77192513aab255871e34a33bc6dc5b6ac925b4908f" exitCode=0
Oct 30 22:08:41.026949 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.026720    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" event={"ID":"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137","Type":"ContainerDied","Data":"d7ab9b793531c68951239f77192513aab255871e34a33bc6dc5b6ac925b4908f"}
Oct 30 22:08:41.027202 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.027160    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk" event={"ID":"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137","Type":"ContainerDied","Data":"230f214f788d45a8e2be73784cdf23c670d47b405c63674547751a5ec3f5538a"}
Oct 30 22:08:41.027407 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.027392    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="230f214f788d45a8e2be73784cdf23c670d47b405c63674547751a5ec3f5538a"
Oct 30 22:08:41.029786 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.029765    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 22:08:41.133843 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.133819    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-socket\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.133956 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.133872    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-data\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.133956 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.133896    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-kube-api-access-gxrdt\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-kube-api-access-gxrdt\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.134054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.133991    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-certs\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.134054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134032    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-socket" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:41.134166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134042    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istiod-ca-cert\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.134166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134144    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-credential-socket\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.134269 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134175    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-data" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:41.134269 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134223    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-podinfo\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.134269 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134264    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-envoy\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.134443 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134306    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-token\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-token\") pod \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\" (UID: \"a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137\") "
Oct 30 22:08:41.134443 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134311    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-certs" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:41.134559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134438    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-credential-socket" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:41.134559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134524    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istiod-ca-cert" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:41.134927 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134904    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.135033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134931    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.135033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134946    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.135033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134959    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.135033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.134972    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.136472 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.136444    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-podinfo" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:41.136568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.136488    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-token" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:41.136733 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.136712    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-envoy" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:41.136845 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.136829    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-kube-api-access-gxrdt" pod "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" (UID: "a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137"). InnerVolumeSpecName "kube-api-access-gxrdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:41.237258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.237189    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.237258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.237220    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.237258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.237231    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:41.237258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:41.237240    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxrdt\" (UniqueName: \"kubernetes.io/projected/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137-kube-api-access-gxrdt\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.034326 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:42.034295    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"
Oct 30 22:08:42.259776 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:42.259745    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"]
Oct 30 22:08:42.311386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:42.311330    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk"]
Oct 30 22:08:43.847937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:43.847904    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" path="/var/lib/kubelet/pods/a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137/volumes"
Oct 30 22:08:50.761290 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:50.761244    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"]
Oct 30 22:08:50.761745 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:50.761499    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" containerID="cri-o://8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921" gracePeriod=30
Oct 30 22:08:51.849373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:51.849289    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"]
Oct 30 22:08:51.849763 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:51.849502    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" containerID="cri-o://0cacb3319df5c254ff37f9e7f82b44054d71ad0025df5a2d1347a9d15101f956" gracePeriod=30
Oct 30 22:08:51.895810 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:51.895778    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"]
Oct 30 22:08:51.896005 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:51.895974    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" containerID="cri-o://95f61289b121034e87c4a084d5c7a818b9bfaeb6aa69834c70b043871a141c46" gracePeriod=30
Oct 30 22:08:52.982745 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:08:52.982705    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:08:53.228157 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:53.228117    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:08:53.228346 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:53.228174    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:55.859659 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:55.859628    5561 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:08:55.860027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:55.859689    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:08:55.860027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:55.859766    5561 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:08:55.860027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:55.859801    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:56.036437 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.036384    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 22:08:56.144070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.144025    5561 generic.go:356] "Generic (PLEG): container finished" podID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerID="8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921" exitCode=0
Oct 30 22:08:56.144070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.144063    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"
Oct 30 22:08:56.144276 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.144082    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" event={"ID":"bba9bb7d-753c-4b57-93b6-0984dac30f06","Type":"ContainerDied","Data":"8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921"}
Oct 30 22:08:56.144276 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.144134    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp" event={"ID":"bba9bb7d-753c-4b57-93b6-0984dac30f06","Type":"ContainerDied","Data":"61ac9016334dcb48ce2a3e32b79e65fb31d6d77d8eae0278c1bae7f69f77f01b"}
Oct 30 22:08:56.144276 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.144162    5561 scope.go:122] "RemoveContainer" containerID="8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921"
Oct 30 22:08:56.154191 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.154170    5561 scope.go:122] "RemoveContainer" containerID="8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921"
Oct 30 22:08:56.154461 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:08:56.154442    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921\": container with ID starting with 8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921 not found: ID does not exist" containerID="8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921"
Oct 30 22:08:56.154541 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.154469    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921"} err="failed to get container status \"8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921\": rpc error: code = NotFound desc = could not find container \"8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921\": container with ID starting with 8ad88b06dd0c4e13e280ba4efc85764efaa946b8492c4f21b40cd3af907e4921 not found: ID does not exist"
Oct 30 22:08:56.178460 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178439    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-credential-socket\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.178566 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178481    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-envoy\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.178566 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178508    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-certs\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.178566 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178539    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-socket\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.178798 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178753    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-data\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.178798 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178757    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-credential-socket" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:56.178798 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178783    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-certs" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:56.179035 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178792    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-kube-api-access-8x5dw\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-kube-api-access-8x5dw\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.179035 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178936    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-socket" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:56.179035 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178956    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-data" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:56.179035 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.178983    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bba9bb7d-753c-4b57-93b6-0984dac30f06-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba9bb7d-753c-4b57-93b6-0984dac30f06-istiod-ca-cert\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.179035 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179019    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-podinfo\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.179302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179052    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-token\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-token\") pod \"bba9bb7d-753c-4b57-93b6-0984dac30f06\" (UID: \"bba9bb7d-753c-4b57-93b6-0984dac30f06\") "
Oct 30 22:08:56.179408 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179312    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba9bb7d-753c-4b57-93b6-0984dac30f06-istiod-ca-cert" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:56.179778 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179757    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.179778 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179782    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba9bb7d-753c-4b57-93b6-0984dac30f06-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.179936 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179796    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.179936 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179809    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.179936 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.179822    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.180852 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.180828    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-envoy" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:56.181432 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.181412    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-token" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:56.181686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.181661    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-kube-api-access-8x5dw" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "kube-api-access-8x5dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:56.183167 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.183147    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-podinfo" pod "bba9bb7d-753c-4b57-93b6-0984dac30f06" (UID: "bba9bb7d-753c-4b57-93b6-0984dac30f06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:56.281426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.281392    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.281426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.281424    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.281426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.281433    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba9bb7d-753c-4b57-93b6-0984dac30f06-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.281651 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.281444    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x5dw\" (UniqueName: \"kubernetes.io/projected/bba9bb7d-753c-4b57-93b6-0984dac30f06-kube-api-access-8x5dw\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:56.571242 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.571201    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"]
Oct 30 22:08:56.620240 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:56.620207    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp"]
Oct 30 22:08:57.164922 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.164893    5561 generic.go:356] "Generic (PLEG): container finished" podID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerID="95f61289b121034e87c4a084d5c7a818b9bfaeb6aa69834c70b043871a141c46" exitCode=0
Oct 30 22:08:57.165305 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.164972    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" event={"ID":"bd298109-a5d1-4b77-a486-fd147afff1d6","Type":"ContainerDied","Data":"95f61289b121034e87c4a084d5c7a818b9bfaeb6aa69834c70b043871a141c46"}
Oct 30 22:08:57.167983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.167855    5561 generic.go:356] "Generic (PLEG): container finished" podID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerID="0cacb3319df5c254ff37f9e7f82b44054d71ad0025df5a2d1347a9d15101f956" exitCode=0
Oct 30 22:08:57.167983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.167911    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" event={"ID":"25dd90d4-46db-4987-8bf5-4ef74c025021","Type":"ContainerDied","Data":"0cacb3319df5c254ff37f9e7f82b44054d71ad0025df5a2d1347a9d15101f956"}
Oct 30 22:08:57.167983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.167946    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh" event={"ID":"25dd90d4-46db-4987-8bf5-4ef74c025021","Type":"ContainerDied","Data":"4b886fd4b6edd86a78f66c3cc88abc4760faa1b1e50a6433c100471f5410fbc7"}
Oct 30 22:08:57.167983 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.167968    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b886fd4b6edd86a78f66c3cc88abc4760faa1b1e50a6433c100471f5410fbc7"
Oct 30 22:08:57.172274 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.172206    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 22:08:57.221915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.221895    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 22:08:57.294868 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.294837    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd298109-a5d1-4b77-a486-fd147afff1d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd298109-a5d1-4b77-a486-fd147afff1d6-istiod-ca-cert\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.294976 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.294880    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-podinfo\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.294976 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.294913    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25dd90d4-46db-4987-8bf5-4ef74c025021-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25dd90d4-46db-4987-8bf5-4ef74c025021-istiod-ca-cert\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.294976 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.294951    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-credential-socket\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.295149 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.294984    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-kube-api-access-rqdfb\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-kube-api-access-rqdfb\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.295149 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295080    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-certs\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.295248 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295152    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-socket\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.295248 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295188    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-kube-api-access-w98vc\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-kube-api-access-w98vc\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.295248 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295240    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-data\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.295425 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295245    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd298109-a5d1-4b77-a486-fd147afff1d6-istiod-ca-cert" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:57.295484 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295421    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25dd90d4-46db-4987-8bf5-4ef74c025021-istiod-ca-cert" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:57.295792 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295667    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-socket" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.295792 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295752    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-credential-socket" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.295792 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295771    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-certs\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295851    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-token\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-token\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295881    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-data\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295774    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-certs" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295945    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-envoy\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295951    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-data" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295988    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-token\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-token\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.295998    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-certs" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296017    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-envoy\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296055    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-socket\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296095    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-podinfo\") pod \"25dd90d4-46db-4987-8bf5-4ef74c025021\" (UID: \"25dd90d4-46db-4987-8bf5-4ef74c025021\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296134    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-credential-socket\") pod \"bd298109-a5d1-4b77-a486-fd147afff1d6\" (UID: \"bd298109-a5d1-4b77-a486-fd147afff1d6\") "
Oct 30 22:08:57.296445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296182    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-data" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296802    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296820    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296835    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296850    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd298109-a5d1-4b77-a486-fd147afff1d6-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296863    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25dd90d4-46db-4987-8bf5-4ef74c025021-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296877    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296890    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.296903    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.297154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.297097    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-credential-socket" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.297765 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.297281    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-socket" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.297765 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.297671    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-podinfo" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:57.297905 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.297880    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-kube-api-access-w98vc" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "kube-api-access-w98vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:57.298235 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.298214    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-kube-api-access-rqdfb" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "kube-api-access-rqdfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:57.298377 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.298338    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-token" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:57.299388 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.299367    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-podinfo" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:57.299843 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.299823    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-envoy" pod "bd298109-a5d1-4b77-a486-fd147afff1d6" (UID: "bd298109-a5d1-4b77-a486-fd147afff1d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.299947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.299932    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-token" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:57.299992 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.299941    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-envoy" pod "25dd90d4-46db-4987-8bf5-4ef74c025021" (UID: "25dd90d4-46db-4987-8bf5-4ef74c025021"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:57.398187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398104    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w98vc\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-kube-api-access-w98vc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398135    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398150    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398162    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398174    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398187    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25dd90d4-46db-4987-8bf5-4ef74c025021-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398200    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25dd90d4-46db-4987-8bf5-4ef74c025021-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398212    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd298109-a5d1-4b77-a486-fd147afff1d6-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398224    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd298109-a5d1-4b77-a486-fd147afff1d6-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.398523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.398235    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqdfb\" (UniqueName: \"kubernetes.io/projected/bd298109-a5d1-4b77-a486-fd147afff1d6-kube-api-access-rqdfb\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:57.847405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:57.847315    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" path="/var/lib/kubelet/pods/bba9bb7d-753c-4b57-93b6-0984dac30f06/volumes"
Oct 30 22:08:58.176223 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.176195    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"
Oct 30 22:08:58.176692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.176225    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4" event={"ID":"bd298109-a5d1-4b77-a486-fd147afff1d6","Type":"ContainerDied","Data":"44b8322ec63e9ea5dbe16a10d7e67afc6b29c2b971b12fdee2741954a22fa096"}
Oct 30 22:08:58.176692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.176286    5561 scope.go:122] "RemoveContainer" containerID="95f61289b121034e87c4a084d5c7a818b9bfaeb6aa69834c70b043871a141c46"
Oct 30 22:08:58.176692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.176532    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"
Oct 30 22:08:58.343695 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.343668    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"]
Oct 30 22:08:58.393398 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.393372    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh"]
Oct 30 22:08:58.457915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.457850    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"]
Oct 30 22:08:58.461083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:58.461064    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4"]
Oct 30 22:08:59.846881 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:59.846843    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" path="/var/lib/kubelet/pods/25dd90d4-46db-4987-8bf5-4ef74c025021/volumes"
Oct 30 22:08:59.847414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:08:59.847396    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" path="/var/lib/kubelet/pods/bd298109-a5d1-4b77-a486-fd147afff1d6/volumes"
Oct 30 22:09:01.185598 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:01.185566    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"]
Oct 30 22:09:01.294463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:01.294431    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"]
Oct 30 22:09:06.187679 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.187627    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t" podUID="729e775b-1219-4eb8-a091-afe81bb48455" containerName="payload-processing" containerID="cri-o://9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a" gracePeriod=25
Oct 30 22:09:06.295201 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.295158    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n" podUID="88f86ca2-a523-428c-b6ad-57c1a63ec4dc" containerName="payload-pre-processing" containerID="cri-o://fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30" gracePeriod=25
Oct 30 22:09:06.574519 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.574492    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:09:06.624621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.624595    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:09:06.656542 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.656516    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-kube-api-access-mhdx8\" (UniqueName: \"kubernetes.io/projected/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-kube-api-access-mhdx8\") pod \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") "
Oct 30 22:09:06.656660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.656589    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/729e775b-1219-4eb8-a091-afe81bb48455-kube-api-access-9tx9j\" (UniqueName: \"kubernetes.io/projected/729e775b-1219-4eb8-a091-afe81bb48455-kube-api-access-9tx9j\") pod \"729e775b-1219-4eb8-a091-afe81bb48455\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") "
Oct 30 22:09:06.656660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.656613    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-plugins-config-volume\") pod \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\" (UID: \"88f86ca2-a523-428c-b6ad-57c1a63ec4dc\") "
Oct 30 22:09:06.656660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.656636    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/729e775b-1219-4eb8-a091-afe81bb48455-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729e775b-1219-4eb8-a091-afe81bb48455-plugins-config-volume\") pod \"729e775b-1219-4eb8-a091-afe81bb48455\" (UID: \"729e775b-1219-4eb8-a091-afe81bb48455\") "
Oct 30 22:09:06.657014 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.656988    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-plugins-config-volume" pod "88f86ca2-a523-428c-b6ad-57c1a63ec4dc" (UID: "88f86ca2-a523-428c-b6ad-57c1a63ec4dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:06.657083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.657037    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/729e775b-1219-4eb8-a091-afe81bb48455-plugins-config-volume" pod "729e775b-1219-4eb8-a091-afe81bb48455" (UID: "729e775b-1219-4eb8-a091-afe81bb48455"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:06.657472 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.657451    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:06.657609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.657474    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729e775b-1219-4eb8-a091-afe81bb48455-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:06.659016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.658994    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-kube-api-access-mhdx8" pod "88f86ca2-a523-428c-b6ad-57c1a63ec4dc" (UID: "88f86ca2-a523-428c-b6ad-57c1a63ec4dc"). InnerVolumeSpecName "kube-api-access-mhdx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:06.659118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.659026    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729e775b-1219-4eb8-a091-afe81bb48455-kube-api-access-9tx9j" pod "729e775b-1219-4eb8-a091-afe81bb48455" (UID: "729e775b-1219-4eb8-a091-afe81bb48455"). InnerVolumeSpecName "kube-api-access-9tx9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:06.758403 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.758318    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhdx8\" (UniqueName: \"kubernetes.io/projected/88f86ca2-a523-428c-b6ad-57c1a63ec4dc-kube-api-access-mhdx8\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:06.758403 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:06.758344    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tx9j\" (UniqueName: \"kubernetes.io/projected/729e775b-1219-4eb8-a091-afe81bb48455-kube-api-access-9tx9j\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:07.269795 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.269764    5561 generic.go:356] "Generic (PLEG): container finished" podID="729e775b-1219-4eb8-a091-afe81bb48455" containerID="9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a" exitCode=0
Oct 30 22:09:07.270204 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.269818    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"
Oct 30 22:09:07.270204 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.269850    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t" event={"ID":"729e775b-1219-4eb8-a091-afe81bb48455","Type":"ContainerDied","Data":"9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a"}
Oct 30 22:09:07.270204 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.269936    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t" event={"ID":"729e775b-1219-4eb8-a091-afe81bb48455","Type":"ContainerDied","Data":"98b045083f2451de55372a3c8874e016c055725c875e15c7f011a9f8bf6cf7ba"}
Oct 30 22:09:07.270204 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.269953    5561 scope.go:122] "RemoveContainer" containerID="9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a"
Oct 30 22:09:07.271229 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.271176    5561 generic.go:356] "Generic (PLEG): container finished" podID="88f86ca2-a523-428c-b6ad-57c1a63ec4dc" containerID="fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30" exitCode=0
Oct 30 22:09:07.271229 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.271208    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n" event={"ID":"88f86ca2-a523-428c-b6ad-57c1a63ec4dc","Type":"ContainerDied","Data":"fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30"}
Oct 30 22:09:07.271393 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.271244    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"
Oct 30 22:09:07.271393 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.271258    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n" event={"ID":"88f86ca2-a523-428c-b6ad-57c1a63ec4dc","Type":"ContainerDied","Data":"f5160c5175d4f25862558cfe4c07d232e31d8ed4c565edfc1a301e1548b0b4cf"}
Oct 30 22:09:07.282004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.281985    5561 scope.go:122] "RemoveContainer" containerID="9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a"
Oct 30 22:09:07.282270 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:09:07.282250    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a\": container with ID starting with 9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a not found: ID does not exist" containerID="9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a"
Oct 30 22:09:07.282312 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.282278    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a"} err="failed to get container status \"9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a\": rpc error: code = NotFound desc = could not find container \"9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a\": container with ID starting with 9f9d5b3c70c5278a2d7ee3bedd9ea812f5d4da58d123bbb3a38ca9f9dc59d08a not found: ID does not exist"
Oct 30 22:09:07.282312 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.282296    5561 scope.go:122] "RemoveContainer" containerID="fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30"
Oct 30 22:09:07.293342 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.293311    5561 scope.go:122] "RemoveContainer" containerID="fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30"
Oct 30 22:09:07.293628 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:09:07.293607    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30\": container with ID starting with fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30 not found: ID does not exist" containerID="fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30"
Oct 30 22:09:07.293693 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.293634    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30"} err="failed to get container status \"fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30\": rpc error: code = NotFound desc = could not find container \"fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30\": container with ID starting with fc21b1cd452bd59827e2a25b477cfe9c01a2ac87f3c63ecf50240e9a8dac6e30 not found: ID does not exist"
Oct 30 22:09:07.452540 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.452505    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"]
Oct 30 22:09:07.500436 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.500364    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n"]
Oct 30 22:09:07.564426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.564391    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"]
Oct 30 22:09:07.570549 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.570523    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t"]
Oct 30 22:09:07.847287 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.847217    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="729e775b-1219-4eb8-a091-afe81bb48455" path="/var/lib/kubelet/pods/729e775b-1219-4eb8-a091-afe81bb48455/volumes"
Oct 30 22:09:07.847703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:07.847687    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88f86ca2-a523-428c-b6ad-57c1a63ec4dc" path="/var/lib/kubelet/pods/88f86ca2-a523-428c-b6ad-57c1a63ec4dc/volumes"
Oct 30 22:09:12.278047 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:12.278014    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:09:12.322905 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:12.322882    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:09:12.475860 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:12.475827    5561 scope.go:122] "RemoveContainer" containerID="0cacb3319df5c254ff37f9e7f82b44054d71ad0025df5a2d1347a9d15101f956"
Oct 30 22:09:12.486114 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:12.486085    5561 scope.go:122] "RemoveContainer" containerID="d7ab9b793531c68951239f77192513aab255871e34a33bc6dc5b6ac925b4908f"
Oct 30 22:09:16.819183 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:16.819150    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:09:22.981213 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:09:22.981172    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:09:27.286989 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:27.286954    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"]
Oct 30 22:09:27.287550 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:27.287172    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" containerID="cri-o://6886a9a49425d81b6cf1965ef946845986cd7dc9a065f7b452e00bfd1ac85a2a" gracePeriod=30
Oct 30 22:09:29.969414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:29.969346    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:09:29.969803 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:29.969441    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:32.487772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.487733    5561 generic.go:356] "Generic (PLEG): container finished" podID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerID="6886a9a49425d81b6cf1965ef946845986cd7dc9a065f7b452e00bfd1ac85a2a" exitCode=0
Oct 30 22:09:32.488257 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.487833    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" event={"ID":"92248b6a-b346-406f-abe3-d3cc80802e1f","Type":"ContainerDied","Data":"6886a9a49425d81b6cf1965ef946845986cd7dc9a065f7b452e00bfd1ac85a2a"}
Oct 30 22:09:32.560020 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.559967    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:09:32.648612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648587    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-credential-socket\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.648748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648708    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-kube-api-access-28sxp\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-kube-api-access-28sxp\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.648792 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648762    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-socket\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.648854 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648829    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-token\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-token\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.648890 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648882    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-podinfo\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.648937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648840    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-credential-socket" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.648937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648927    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-data\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.649025 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.648963    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92248b6a-b346-406f-abe3-d3cc80802e1f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92248b6a-b346-406f-abe3-d3cc80802e1f-istiod-ca-cert\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.649081 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649022    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-envoy\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.649138 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649076    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-certs\") pod \"92248b6a-b346-406f-abe3-d3cc80802e1f\" (UID: \"92248b6a-b346-406f-abe3-d3cc80802e1f\") "
Oct 30 22:09:32.649361 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649074    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-socket" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.649492 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649247    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-data" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.649576 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649432    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-certs" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.649636 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649571    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92248b6a-b346-406f-abe3-d3cc80802e1f-istiod-ca-cert" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:32.649954 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649926    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.650040 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649960    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92248b6a-b346-406f-abe3-d3cc80802e1f-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.650040 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649978    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.650040 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.649993    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.650040 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.650009    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.651037 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.651012    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-token" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:32.651119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.651037    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-podinfo" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:32.651325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.651301    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-envoy" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.651460 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.651334    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-kube-api-access-28sxp" pod "92248b6a-b346-406f-abe3-d3cc80802e1f" (UID: "92248b6a-b346-406f-abe3-d3cc80802e1f"). InnerVolumeSpecName "kube-api-access-28sxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:32.751647 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.751613    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.751647 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.751648    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28sxp\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-kube-api-access-28sxp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.751885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.751664    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.751885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:32.751678    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92248b6a-b346-406f-abe3-d3cc80802e1f-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.495514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:33.495479    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"
Oct 30 22:09:33.495514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:33.495498    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc" event={"ID":"92248b6a-b346-406f-abe3-d3cc80802e1f","Type":"ContainerDied","Data":"17b2abd903d7f4fa698bbbb25d6d4cc1482f0eecd7436476bba29062b3f4577f"}
Oct 30 22:09:33.496083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:33.495564    5561 scope.go:122] "RemoveContainer" containerID="6886a9a49425d81b6cf1965ef946845986cd7dc9a065f7b452e00bfd1ac85a2a"
Oct 30 22:09:33.712364 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:33.712315    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"]
Oct 30 22:09:33.762730 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:33.762659    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc"]
Oct 30 22:09:33.847567 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:33.847507    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" path="/var/lib/kubelet/pods/92248b6a-b346-406f-abe3-d3cc80802e1f/volumes"
Oct 30 22:09:36.220131 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:36.220085    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"]
Oct 30 22:09:36.220613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:36.220330    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" containerID="cri-o://3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa" gracePeriod=30
Oct 30 22:09:38.258188 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:38.258150    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:09:38.258644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:38.258223    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:41.499672 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.499617    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 22:09:41.515971 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.515942    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-podinfo\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516111 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516007    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-kube-api-access-zwqk5\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-kube-api-access-zwqk5\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516111 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516040    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-data\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516111 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516067    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-envoy\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516111 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516092    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-token\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-token\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516135    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-certs\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516164    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-socket\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516378    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-credential-socket\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516376    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-data" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:41.516497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516483    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istiod-ca-cert\") pod \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\" (UID: \"b25b0f19-8746-49b5-a2a0-d9ad45116d6a\") "
Oct 30 22:09:41.516653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516512    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-certs" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:41.516653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516582    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-socket" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:41.516774 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516712    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-credential-socket" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:41.516985 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.516963    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istiod-ca-cert" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:41.517197 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.517181    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.517272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.517199    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.517272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.517210    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.517272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.517220    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.517272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.517229    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.518296 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.518274    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-envoy" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:41.518418 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.518372    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-podinfo" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:41.518668 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.518645    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-kube-api-access-zwqk5" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "kube-api-access-zwqk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:41.518803 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.518787    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-token" pod "b25b0f19-8746-49b5-a2a0-d9ad45116d6a" (UID: "b25b0f19-8746-49b5-a2a0-d9ad45116d6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:41.550997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.550970    5561 generic.go:356] "Generic (PLEG): container finished" podID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerID="3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa" exitCode=0
Oct 30 22:09:41.551091 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.551003    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" event={"ID":"b25b0f19-8746-49b5-a2a0-d9ad45116d6a","Type":"ContainerDied","Data":"3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa"}
Oct 30 22:09:41.551091 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.551019    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"
Oct 30 22:09:41.551091 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.551057    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm" event={"ID":"b25b0f19-8746-49b5-a2a0-d9ad45116d6a","Type":"ContainerDied","Data":"de8f0a1bd158e4b38510505eb8bbcfbad5fa7321643d6fd6338e7e07db2f02ca"}
Oct 30 22:09:41.551091 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.551081    5561 scope.go:122] "RemoveContainer" containerID="3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa"
Oct 30 22:09:41.562141 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.562120    5561 scope.go:122] "RemoveContainer" containerID="3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa"
Oct 30 22:09:41.562438 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:09:41.562418    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa\": container with ID starting with 3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa not found: ID does not exist" containerID="3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa"
Oct 30 22:09:41.562485 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.562446    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa"} err="failed to get container status \"3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa\": rpc error: code = NotFound desc = could not find container \"3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa\": container with ID starting with 3dbb87ee79ea603551d0198ef2a42a79f7480ebc35baa102e4ed98175b0b2ffa not found: ID does not exist"
Oct 30 22:09:41.619033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.619009    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.619033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.619029    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwqk5\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-kube-api-access-zwqk5\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.619160 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.619041    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.619160 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.619050    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25b0f19-8746-49b5-a2a0-d9ad45116d6a-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:41.790924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.790839    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"]
Oct 30 22:09:41.793604 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.793577    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm"]
Oct 30 22:09:41.846887 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:41.846856    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" path="/var/lib/kubelet/pods/b25b0f19-8746-49b5-a2a0-d9ad45116d6a/volumes"
Oct 30 22:09:50.553400 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:50.553364    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"]
Oct 30 22:09:50.660099 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:50.660068    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"]
Oct 30 22:09:52.984225 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:09:52.984191    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:09:55.553991 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.553950    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd" podUID="6fcbc277-8220-4a5e-865e-1bba7d499d29" containerName="payload-processing" containerID="cri-o://3e443ba3e855cfb0fb57e0464888eba24b088e4f8effbd063e1766a4b4dcabce" gracePeriod=25
Oct 30 22:09:55.661079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.661036    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n" podUID="b6f06cd4-015e-4e94-aaaf-0bde5c12d321" containerName="payload-pre-processing" containerID="cri-o://fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6" gracePeriod=25
Oct 30 22:09:55.689555 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.689527    5561 generic.go:356] "Generic (PLEG): container finished" podID="6fcbc277-8220-4a5e-865e-1bba7d499d29" containerID="3e443ba3e855cfb0fb57e0464888eba24b088e4f8effbd063e1766a4b4dcabce" exitCode=0
Oct 30 22:09:55.689656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.689580    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd" event={"ID":"6fcbc277-8220-4a5e-865e-1bba7d499d29","Type":"ContainerDied","Data":"3e443ba3e855cfb0fb57e0464888eba24b088e4f8effbd063e1766a4b4dcabce"}
Oct 30 22:09:55.855875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.855840    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:09:55.915027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.914994    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fcbc277-8220-4a5e-865e-1bba7d499d29-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fcbc277-8220-4a5e-865e-1bba7d499d29-plugins-config-volume\") pod \"6fcbc277-8220-4a5e-865e-1bba7d499d29\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") "
Oct 30 22:09:55.915237 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.915217    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fcbc277-8220-4a5e-865e-1bba7d499d29-kube-api-access-dkz5n\" (UniqueName: \"kubernetes.io/projected/6fcbc277-8220-4a5e-865e-1bba7d499d29-kube-api-access-dkz5n\") pod \"6fcbc277-8220-4a5e-865e-1bba7d499d29\" (UID: \"6fcbc277-8220-4a5e-865e-1bba7d499d29\") "
Oct 30 22:09:55.915486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.915457    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fcbc277-8220-4a5e-865e-1bba7d499d29-plugins-config-volume" pod "6fcbc277-8220-4a5e-865e-1bba7d499d29" (UID: "6fcbc277-8220-4a5e-865e-1bba7d499d29"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:55.916277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.916117    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fcbc277-8220-4a5e-865e-1bba7d499d29-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:55.918899 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.918869    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fcbc277-8220-4a5e-865e-1bba7d499d29-kube-api-access-dkz5n" pod "6fcbc277-8220-4a5e-865e-1bba7d499d29" (UID: "6fcbc277-8220-4a5e-865e-1bba7d499d29"). InnerVolumeSpecName "kube-api-access-dkz5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:55.955753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:55.955733    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:09:56.016731 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.016705    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-plugins-config-volume\") pod \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") "
Oct 30 22:09:56.016882 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.016738    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-kube-api-access-tv9xz\" (UniqueName: \"kubernetes.io/projected/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-kube-api-access-tv9xz\") pod \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\" (UID: \"b6f06cd4-015e-4e94-aaaf-0bde5c12d321\") "
Oct 30 22:09:56.017106 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.017075    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-plugins-config-volume" pod "b6f06cd4-015e-4e94-aaaf-0bde5c12d321" (UID: "b6f06cd4-015e-4e94-aaaf-0bde5c12d321"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:56.017343 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.017323    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkz5n\" (UniqueName: \"kubernetes.io/projected/6fcbc277-8220-4a5e-865e-1bba7d499d29-kube-api-access-dkz5n\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:56.017446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.017345    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:56.018945 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.018919    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-kube-api-access-tv9xz" pod "b6f06cd4-015e-4e94-aaaf-0bde5c12d321" (UID: "b6f06cd4-015e-4e94-aaaf-0bde5c12d321"). InnerVolumeSpecName "kube-api-access-tv9xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:56.120437 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.120398    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv9xz\" (UniqueName: \"kubernetes.io/projected/b6f06cd4-015e-4e94-aaaf-0bde5c12d321-kube-api-access-tv9xz\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:56.697314 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.697276    5561 generic.go:356] "Generic (PLEG): container finished" podID="b6f06cd4-015e-4e94-aaaf-0bde5c12d321" containerID="fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6" exitCode=0
Oct 30 22:09:56.697788 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.697334    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"
Oct 30 22:09:56.697788 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.697383    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n" event={"ID":"b6f06cd4-015e-4e94-aaaf-0bde5c12d321","Type":"ContainerDied","Data":"fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6"}
Oct 30 22:09:56.697788 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.697471    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n" event={"ID":"b6f06cd4-015e-4e94-aaaf-0bde5c12d321","Type":"ContainerDied","Data":"9f28ab0a245a0ce47cb397856f6459edf275016c0574155d4c83b5e149819343"}
Oct 30 22:09:56.697788 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.697496    5561 scope.go:122] "RemoveContainer" containerID="fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6"
Oct 30 22:09:56.699004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.698981    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd" event={"ID":"6fcbc277-8220-4a5e-865e-1bba7d499d29","Type":"ContainerDied","Data":"fe5579586845c5f40e0ecafbaef98582d8a764167702dd599050a3a245ca292a"}
Oct 30 22:09:56.699090 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.699068    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"
Oct 30 22:09:56.708098 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.708077    5561 scope.go:122] "RemoveContainer" containerID="fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6"
Oct 30 22:09:56.708391 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:09:56.708360    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6\": container with ID starting with fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6 not found: ID does not exist" containerID="fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6"
Oct 30 22:09:56.708556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.708401    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6"} err="failed to get container status \"fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6\": rpc error: code = NotFound desc = could not find container \"fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6\": container with ID starting with fd1b12bd10b490f88f713c5ebba828d1c449e0c85dd4cee69498b6903a7a89a6 not found: ID does not exist"
Oct 30 22:09:56.708556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.708417    5561 scope.go:122] "RemoveContainer" containerID="3e443ba3e855cfb0fb57e0464888eba24b088e4f8effbd063e1766a4b4dcabce"
Oct 30 22:09:56.872083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.872054    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"]
Oct 30 22:09:56.922336 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.922308    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd"]
Oct 30 22:09:56.989650 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.989619    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"]
Oct 30 22:09:56.997205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:56.997180    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n"]
Oct 30 22:09:57.848294 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:57.848259    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fcbc277-8220-4a5e-865e-1bba7d499d29" path="/var/lib/kubelet/pods/6fcbc277-8220-4a5e-865e-1bba7d499d29/volumes"
Oct 30 22:09:57.848819 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:09:57.848785    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6f06cd4-015e-4e94-aaaf-0bde5c12d321" path="/var/lib/kubelet/pods/b6f06cd4-015e-4e94-aaaf-0bde5c12d321/volumes"
Oct 30 22:10:09.585565 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:09.585527    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"]
Oct 30 22:10:14.586079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.586037    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4" podUID="6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" containerName="payload-processing" containerID="cri-o://21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1" gracePeriod=25
Oct 30 22:10:14.870390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.870361    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:10:14.891735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.891703    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-plugins-config-volume\") pod \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") "
Oct 30 22:10:14.891898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.891787    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-kube-api-access-4zvtm\" (UniqueName: \"kubernetes.io/projected/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-kube-api-access-4zvtm\") pod \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\" (UID: \"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab\") "
Oct 30 22:10:14.892090 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.892064    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-plugins-config-volume" pod "6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" (UID: "6ce6b173-dc1b-4f8a-817a-0299b3e8fbab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:14.893778 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.893520    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:14.897805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.897781    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-kube-api-access-4zvtm" pod "6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" (UID: "6ce6b173-dc1b-4f8a-817a-0299b3e8fbab"). InnerVolumeSpecName "kube-api-access-4zvtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:14.959594 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.959565    5561 generic.go:356] "Generic (PLEG): container finished" podID="6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" containerID="21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1" exitCode=0
Oct 30 22:10:14.959722 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.959622    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"
Oct 30 22:10:14.959722 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.959633    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4" event={"ID":"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab","Type":"ContainerDied","Data":"21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1"}
Oct 30 22:10:14.959722 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.959688    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4" event={"ID":"6ce6b173-dc1b-4f8a-817a-0299b3e8fbab","Type":"ContainerDied","Data":"b6fa8874fc6bde30052f9060bf9f41ca88b2215dcff9bd4b4eb33c67f8dff929"}
Oct 30 22:10:14.959722 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.959722    5561 scope.go:122] "RemoveContainer" containerID="21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1"
Oct 30 22:10:14.972417 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.972395    5561 scope.go:122] "RemoveContainer" containerID="21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1"
Oct 30 22:10:14.978654 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:14.978626    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1\": container with ID starting with 21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1 not found: ID does not exist" containerID="21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1"
Oct 30 22:10:14.978739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.978664    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1"} err="failed to get container status \"21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1\": rpc error: code = NotFound desc = could not find container \"21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1\": container with ID starting with 21d8ac6d6d2e932b759ad18e1a4dbde475d7bb4fb7cbd91665c9feffd813a4c1 not found: ID does not exist"
Oct 30 22:10:14.994979 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:14.994950    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zvtm\" (UniqueName: \"kubernetes.io/projected/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab-kube-api-access-4zvtm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:15.119117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:15.119068    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"]
Oct 30 22:10:15.121882 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:15.121857    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4"]
Oct 30 22:10:15.846302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:15.846268    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" path="/var/lib/kubelet/pods/6ce6b173-dc1b-4f8a-817a-0299b3e8fbab/volumes"
Oct 30 22:10:22.291155 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:22.291069    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"]
Oct 30 22:10:22.291643 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:22.291297    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy" containerID="cri-o://43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d" gracePeriod=30
Oct 30 22:10:22.984791 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:22.984754    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:10:23.532860 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:23.532829    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"]
Oct 30 22:10:23.533247 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:23.533035    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy" containerID="cri-o://4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99" gracePeriod=30
Oct 30 22:10:26.610977 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:26.610942    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"]
Oct 30 22:10:26.718057 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:26.718018    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"]
Oct 30 22:10:27.554088 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.554027    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 22:10:27.623944 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.623909    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-certs\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.623953    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3b174d9-930f-4553-b456-cf975d3612f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b174d9-930f-4553-b456-cf975d3612f8-istiod-ca-cert\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.623984    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-socket\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624037    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-data\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624149    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-credential-socket\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624190    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-envoy\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624223    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-certs" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624257    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-kube-api-access-5hpbg\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-kube-api-access-5hpbg\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624375    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-istio-token\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-istio-token\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624396    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-socket" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.624430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624428    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b174d9-930f-4553-b456-cf975d3612f8-istiod-ca-cert" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:27.624947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624444    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-credential-socket" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.624947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624469    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3b174d9-930f-4553-b456-cf975d3612f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b174d9-930f-4553-b456-cf975d3612f8-istio-podinfo\") pod \"a3b174d9-930f-4553-b456-cf975d3612f8\" (UID: \"a3b174d9-930f-4553-b456-cf975d3612f8\") "
Oct 30 22:10:27.624947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.624756    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-data" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.625156 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.625134    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.625208 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.625162    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b174d9-930f-4553-b456-cf975d3612f8-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.625208 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.625177    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.625208 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.625191    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.625208 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.625204    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.627015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.626988    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-envoy" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.627174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.627021    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-istio-token" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:27.627248 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.627219    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3b174d9-930f-4553-b456-cf975d3612f8-istio-podinfo" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:27.627401 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.627265    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-kube-api-access-5hpbg" pod "a3b174d9-930f-4553-b456-cf975d3612f8" (UID: "a3b174d9-930f-4553-b456-cf975d3612f8"). InnerVolumeSpecName "kube-api-access-5hpbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:27.726864 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.726826    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b174d9-930f-4553-b456-cf975d3612f8-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.726864 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.726865    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hpbg\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-kube-api-access-5hpbg\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.727061 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.726882    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b174d9-930f-4553-b456-cf975d3612f8-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.727061 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:27.726896    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b174d9-930f-4553-b456-cf975d3612f8-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:28.156451 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.155029    5561 generic.go:356] "Generic (PLEG): container finished" podID="a3b174d9-930f-4553-b456-cf975d3612f8" containerID="43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d" exitCode=0
Oct 30 22:10:28.156451 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.155181    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" event={"ID":"a3b174d9-930f-4553-b456-cf975d3612f8","Type":"ContainerDied","Data":"43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d"}
Oct 30 22:10:28.156451 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.155237    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26" event={"ID":"a3b174d9-930f-4553-b456-cf975d3612f8","Type":"ContainerDied","Data":"473065c272a7d8a4b99c0d431f766ef128c93fcdd0ade214e793cbdc0f7501c8"}
Oct 30 22:10:28.156451 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.155265    5561 scope.go:122] "RemoveContainer" containerID="43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d"
Oct 30 22:10:28.156451 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.155454    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"
Oct 30 22:10:28.176714 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.176691    5561 scope.go:122] "RemoveContainer" containerID="43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d"
Oct 30 22:10:28.177041 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:28.177013    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d\": container with ID starting with 43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d not found: ID does not exist" containerID="43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d"
Oct 30 22:10:28.177156 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.177050    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d"} err="failed to get container status \"43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d\": rpc error: code = NotFound desc = could not find container \"43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d\": container with ID starting with 43c84554fea15b040e3e8e62189ca1d38aba71399b9511b1bb4f333ad191c06d not found: ID does not exist"
Oct 30 22:10:28.343435 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.333618    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"]
Oct 30 22:10:28.397761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.395181    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26"]
Oct 30 22:10:28.807284 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.807229    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:10:28.951274 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951235    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-credential-socket\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.951468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951380    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-data\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.951468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951454    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-certs\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.951600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951491    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24ea8511-f145-4af1-95bc-6e10dc90379c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ea8511-f145-4af1-95bc-6e10dc90379c-istiod-ca-cert\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.951600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951522    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-envoy\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.951600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951532    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-credential-socket" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:28.951600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951566    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-kube-api-access-ftdlm\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-kube-api-access-ftdlm\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.951860 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951604    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-token\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-token\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.951860 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951682    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-data" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:28.951994 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.951968    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-socket" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:28.952090 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.952017    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24ea8511-f145-4af1-95bc-6e10dc90379c-istiod-ca-cert" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:28.952090 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.952040    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-socket\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.952201 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.952092    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-certs" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:28.952201 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.952116    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-podinfo\") pod \"24ea8511-f145-4af1-95bc-6e10dc90379c\" (UID: \"24ea8511-f145-4af1-95bc-6e10dc90379c\") "
Oct 30 22:10:28.953015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.952989    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:28.953137 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.953017    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:28.953137 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.953033    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:28.953137 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.953047    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24ea8511-f145-4af1-95bc-6e10dc90379c-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:28.953137 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.953060    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:28.954449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.954419    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-envoy" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:28.955742 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.955720    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-token" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:28.955829 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.955750    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-podinfo" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:28.956170 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:28.956145    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-kube-api-access-ftdlm" pod "24ea8511-f145-4af1-95bc-6e10dc90379c" (UID: "24ea8511-f145-4af1-95bc-6e10dc90379c"). InnerVolumeSpecName "kube-api-access-ftdlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:29.054861 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.054835    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:29.054861 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.054860    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftdlm\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-kube-api-access-ftdlm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:29.055039 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.054872    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:29.055039 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.054881    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24ea8511-f145-4af1-95bc-6e10dc90379c-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:29.163042 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.163010    5561 generic.go:356] "Generic (PLEG): container finished" podID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerID="4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99" exitCode=0
Oct 30 22:10:29.163214 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.163060    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"
Oct 30 22:10:29.163214 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.163097    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" event={"ID":"24ea8511-f145-4af1-95bc-6e10dc90379c","Type":"ContainerDied","Data":"4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99"}
Oct 30 22:10:29.163214 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.163151    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr" event={"ID":"24ea8511-f145-4af1-95bc-6e10dc90379c","Type":"ContainerDied","Data":"68804c84d54b9bafe2c6c12cfbdff3362122826cb4efb983522a9d4c66627a75"}
Oct 30 22:10:29.163214 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.163184    5561 scope.go:122] "RemoveContainer" containerID="4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99"
Oct 30 22:10:29.173515 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.173493    5561 scope.go:122] "RemoveContainer" containerID="3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8"
Oct 30 22:10:29.182496 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.182480    5561 scope.go:122] "RemoveContainer" containerID="4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99"
Oct 30 22:10:29.182752 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:29.182729    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99\": container with ID starting with 4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99 not found: ID does not exist" containerID="4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99"
Oct 30 22:10:29.182816 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.182762    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99"} err="failed to get container status \"4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99\": rpc error: code = NotFound desc = could not find container \"4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99\": container with ID starting with 4758ac5e483dfa442dd3c67d26435fa3e64a54601ef1f5db8e24495467556e99 not found: ID does not exist"
Oct 30 22:10:29.182816 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.182787    5561 scope.go:122] "RemoveContainer" containerID="3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8"
Oct 30 22:10:29.183085 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:29.183053    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8\": container with ID starting with 3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8 not found: ID does not exist" containerID="3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8"
Oct 30 22:10:29.183177 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.183092    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8"} err="failed to get container status \"3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8\": rpc error: code = NotFound desc = could not find container \"3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8\": container with ID starting with 3711e3c5db07aeb76c5729b0c01ff58f984233545fe2212c280fed208f8896f8 not found: ID does not exist"
Oct 30 22:10:29.338063 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.338028    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"]
Oct 30 22:10:29.386303 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.386273    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr"]
Oct 30 22:10:29.846489 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.846457    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" path="/var/lib/kubelet/pods/24ea8511-f145-4af1-95bc-6e10dc90379c/volumes"
Oct 30 22:10:29.847009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:29.846992    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" path="/var/lib/kubelet/pods/a3b174d9-930f-4553-b456-cf975d3612f8/volumes"
Oct 30 22:10:31.612236 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.612190    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5" podUID="9195b46b-8260-4b7a-a879-20c060475ed6" containerName="payload-processing" containerID="cri-o://f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931" gracePeriod=25
Oct 30 22:10:31.718997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.718959    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm" podUID="66a9c077-71c3-468b-b5fd-12d6ba862263" containerName="payload-pre-processing" containerID="cri-o://6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d" gracePeriod=25
Oct 30 22:10:31.896950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.896926    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:10:31.990877 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.990848    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9195b46b-8260-4b7a-a879-20c060475ed6-kube-api-access-xncxg\" (UniqueName: \"kubernetes.io/projected/9195b46b-8260-4b7a-a879-20c060475ed6-kube-api-access-xncxg\") pod \"9195b46b-8260-4b7a-a879-20c060475ed6\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") "
Oct 30 22:10:31.990987 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.990930    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9195b46b-8260-4b7a-a879-20c060475ed6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9195b46b-8260-4b7a-a879-20c060475ed6-plugins-config-volume\") pod \"9195b46b-8260-4b7a-a879-20c060475ed6\" (UID: \"9195b46b-8260-4b7a-a879-20c060475ed6\") "
Oct 30 22:10:31.991335 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.991310    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9195b46b-8260-4b7a-a879-20c060475ed6-plugins-config-volume" pod "9195b46b-8260-4b7a-a879-20c060475ed6" (UID: "9195b46b-8260-4b7a-a879-20c060475ed6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:31.991800 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.991776    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9195b46b-8260-4b7a-a879-20c060475ed6-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:31.992912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.992887    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9195b46b-8260-4b7a-a879-20c060475ed6-kube-api-access-xncxg" pod "9195b46b-8260-4b7a-a879-20c060475ed6" (UID: "9195b46b-8260-4b7a-a879-20c060475ed6"). InnerVolumeSpecName "kube-api-access-xncxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:31.999516 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:31.999492    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:10:32.092984 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.092955    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66a9c077-71c3-468b-b5fd-12d6ba862263-kube-api-access-7flkk\" (UniqueName: \"kubernetes.io/projected/66a9c077-71c3-468b-b5fd-12d6ba862263-kube-api-access-7flkk\") pod \"66a9c077-71c3-468b-b5fd-12d6ba862263\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") "
Oct 30 22:10:32.093116 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.093025    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66a9c077-71c3-468b-b5fd-12d6ba862263-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a9c077-71c3-468b-b5fd-12d6ba862263-plugins-config-volume\") pod \"66a9c077-71c3-468b-b5fd-12d6ba862263\" (UID: \"66a9c077-71c3-468b-b5fd-12d6ba862263\") "
Oct 30 22:10:32.093429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.093396    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a9c077-71c3-468b-b5fd-12d6ba862263-plugins-config-volume" pod "66a9c077-71c3-468b-b5fd-12d6ba862263" (UID: "66a9c077-71c3-468b-b5fd-12d6ba862263"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:32.093576 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.093556    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xncxg\" (UniqueName: \"kubernetes.io/projected/9195b46b-8260-4b7a-a879-20c060475ed6-kube-api-access-xncxg\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:32.093640 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.093579    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a9c077-71c3-468b-b5fd-12d6ba862263-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:32.095016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.094999    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a9c077-71c3-468b-b5fd-12d6ba862263-kube-api-access-7flkk" pod "66a9c077-71c3-468b-b5fd-12d6ba862263" (UID: "66a9c077-71c3-468b-b5fd-12d6ba862263"). InnerVolumeSpecName "kube-api-access-7flkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:32.184310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.184277    5561 generic.go:356] "Generic (PLEG): container finished" podID="66a9c077-71c3-468b-b5fd-12d6ba862263" containerID="6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d" exitCode=0
Oct 30 22:10:32.184497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.184320    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"
Oct 30 22:10:32.184497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.184375    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm" event={"ID":"66a9c077-71c3-468b-b5fd-12d6ba862263","Type":"ContainerDied","Data":"6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d"}
Oct 30 22:10:32.184497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.184425    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm" event={"ID":"66a9c077-71c3-468b-b5fd-12d6ba862263","Type":"ContainerDied","Data":"e3362612a62c2426e2b735ac559a7c604a432328c5ac792d99b4e5a2abf23b0f"}
Oct 30 22:10:32.184497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.184447    5561 scope.go:122] "RemoveContainer" containerID="6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d"
Oct 30 22:10:32.185968 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.185943    5561 generic.go:356] "Generic (PLEG): container finished" podID="9195b46b-8260-4b7a-a879-20c060475ed6" containerID="f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931" exitCode=0
Oct 30 22:10:32.186076 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.185992    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"
Oct 30 22:10:32.186076 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.186032    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5" event={"ID":"9195b46b-8260-4b7a-a879-20c060475ed6","Type":"ContainerDied","Data":"f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931"}
Oct 30 22:10:32.186179 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.186091    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5" event={"ID":"9195b46b-8260-4b7a-a879-20c060475ed6","Type":"ContainerDied","Data":"63cc0708f787281a8827e7b4948b6872b91f80e72dbe535c347a804797933e11"}
Oct 30 22:10:32.194404 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.194380    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7flkk\" (UniqueName: \"kubernetes.io/projected/66a9c077-71c3-468b-b5fd-12d6ba862263-kube-api-access-7flkk\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:32.196409 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.196345    5561 scope.go:122] "RemoveContainer" containerID="6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d"
Oct 30 22:10:32.196659 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:32.196640    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d\": container with ID starting with 6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d not found: ID does not exist" containerID="6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d"
Oct 30 22:10:32.196759 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.196665    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d"} err="failed to get container status \"6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d\": rpc error: code = NotFound desc = could not find container \"6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d\": container with ID starting with 6840150575dd0bc798e824027bb2ce2a508b2e36cfa3575d542d9401713b0d9d not found: ID does not exist"
Oct 30 22:10:32.196759 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.196688    5561 scope.go:122] "RemoveContainer" containerID="f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931"
Oct 30 22:10:32.205089 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.205066    5561 scope.go:122] "RemoveContainer" containerID="f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931"
Oct 30 22:10:32.205338 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:32.205320    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931\": container with ID starting with f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931 not found: ID does not exist" containerID="f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931"
Oct 30 22:10:32.205488 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.205343    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931"} err="failed to get container status \"f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931\": rpc error: code = NotFound desc = could not find container \"f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931\": container with ID starting with f9c5dc34de76aead00017a0765ad203823ba69ad5c8c3fe6a0d7af942cf1f931 not found: ID does not exist"
Oct 30 22:10:32.350236 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.350206    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"]
Oct 30 22:10:32.353838 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.353816    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5"]
Oct 30 22:10:32.463011 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.462946    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"]
Oct 30 22:10:32.466680 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.466658    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm"]
Oct 30 22:10:32.845586 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.845511    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"]
Oct 30 22:10:32.846048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846034    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9195b46b-8260-4b7a-a879-20c060475ed6" containerName="payload-processing"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846049    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9195b46b-8260-4b7a-a879-20c060475ed6" containerName="payload-processing"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846060    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846066    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846075    5561 cpu_manager.go:414] "containerMap: removing container" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846080    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846091    5561 cpu_manager.go:414] "containerMap: removing container" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846095    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846103    5561 cpu_manager.go:414] "containerMap: removing container" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy"
Oct 30 22:10:32.846110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846107    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846121    5561 cpu_manager.go:414] "containerMap: removing container" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846125    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846133    5561 cpu_manager.go:414] "containerMap: removing container" podUID="6fcbc277-8220-4a5e-865e-1bba7d499d29" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846137    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fcbc277-8220-4a5e-865e-1bba7d499d29" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846143    5561 cpu_manager.go:414] "containerMap: removing container" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846147    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846156    5561 cpu_manager.go:414] "containerMap: removing container" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846162    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846168    5561 cpu_manager.go:414] "containerMap: removing container" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846173    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846181    5561 cpu_manager.go:414] "containerMap: removing container" podUID="88f86ca2-a523-428c-b6ad-57c1a63ec4dc" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846186    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88f86ca2-a523-428c-b6ad-57c1a63ec4dc" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846193    5561 cpu_manager.go:414] "containerMap: removing container" podUID="729e775b-1219-4eb8-a091-afe81bb48455" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846197    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="729e775b-1219-4eb8-a091-afe81bb48455" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846202    5561 cpu_manager.go:414] "containerMap: removing container" podUID="6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846207    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846215    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846220    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846229    5561 cpu_manager.go:414] "containerMap: removing container" podUID="66a9c077-71c3-468b-b5fd-12d6ba862263" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846233    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66a9c077-71c3-468b-b5fd-12d6ba862263" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846243    5561 cpu_manager.go:414] "containerMap: removing container" podUID="b6f06cd4-015e-4e94-aaaf-0bde5c12d321" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846248    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6f06cd4-015e-4e94-aaaf-0bde5c12d321" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846259    5561 cpu_manager.go:414] "containerMap: removing container" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846263    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846268    5561 cpu_manager.go:414] "containerMap: removing container" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846273    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846342    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2544bf5-c6ac-4b3a-91f7-4de1f5fd8137" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846363    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="84239b3b-2157-4921-aacd-ed21bc708701" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846370    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba9bb7d-753c-4b57-93b6-0984dac30f06" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846377    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="b25b0f19-8746-49b5-a2a0-d9ad45116d6a" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846385    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d09b71b-828c-4956-9d0b-d2da25d89b50" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846390    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fcbc277-8220-4a5e-865e-1bba7d499d29" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846396    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="25dd90d4-46db-4987-8bf5-4ef74c025021" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846402    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="92248b6a-b346-406f-abe3-d3cc80802e1f" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846407    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846413    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="88f86ca2-a523-428c-b6ad-57c1a63ec4dc" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846418    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="729e775b-1219-4eb8-a091-afe81bb48455" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846424    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ce6b173-dc1b-4f8a-817a-0299b3e8fbab" containerName="payload-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846429    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3b174d9-930f-4553-b456-cf975d3612f8" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846433    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="66a9c077-71c3-468b-b5fd-12d6ba862263" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846440    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6f06cd4-015e-4e94-aaaf-0bde5c12d321" containerName="payload-pre-processing"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846446    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd298109-a5d1-4b77-a486-fd147afff1d6" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846453    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="24ea8511-f145-4af1-95bc-6e10dc90379c" containerName="istio-proxy"
Oct 30 22:10:32.846463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.846460    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9195b46b-8260-4b7a-a879-20c060475ed6" containerName="payload-processing"
Oct 30 22:10:32.850929 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.850912    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:32.936997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.936968    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-9da607\""
Oct 30 22:10:32.937604 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.937417    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-9da607-dockercfg-jkdt6\""
Oct 30 22:10:32.969298 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.969260    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55852795-178a-4281-b268-eff4af7382d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:32.969817 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.969786    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkw4\" (UniqueName: \"kubernetes.io/projected/55852795-178a-4281-b268-eff4af7382d2-kube-api-access-grkw4\") pod \"payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:32.972403 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:32.972378    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"]
Oct 30 22:10:33.043241 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.043189    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"]
Oct 30 22:10:33.052059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.051270    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.069478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.069450    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"]
Oct 30 22:10:33.071103 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.071076    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62f24f9-d580-474f-a109-ff1c9e879bcf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.071217 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.071194    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkw4\" (UniqueName: \"kubernetes.io/projected/55852795-178a-4281-b268-eff4af7382d2-kube-api-access-grkw4\") pod \"payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:33.071327 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.071314    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55852795-178a-4281-b268-eff4af7382d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:33.071490 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.071470    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shknj\" (UniqueName: \"kubernetes.io/projected/e62f24f9-d580-474f-a109-ff1c9e879bcf-kube-api-access-shknj\") pod \"payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.072424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.072393    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55852795-178a-4281-b268-eff4af7382d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:33.129608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.129406    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkw4\" (UniqueName: \"kubernetes.io/projected/55852795-178a-4281-b268-eff4af7382d2-kube-api-access-grkw4\") pod \"payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:33.165666 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.165642    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:33.172395 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.172343    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shknj\" (UniqueName: \"kubernetes.io/projected/e62f24f9-d580-474f-a109-ff1c9e879bcf-kube-api-access-shknj\") pod \"payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.172509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.172424    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62f24f9-d580-474f-a109-ff1c9e879bcf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.173086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.173065    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62f24f9-d580-474f-a109-ff1c9e879bcf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.243503 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.239142    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shknj\" (UniqueName: \"kubernetes.io/projected/e62f24f9-d580-474f-a109-ff1c9e879bcf-kube-api-access-shknj\") pod \"payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.369954 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.369925    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:33.543692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.543667    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"]
Oct 30 22:10:33.544552 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:10:33.544526    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55852795_178a_4281_b268_eff4af7382d2.slice/crio-7aaa587147681d36b5b21917d898dba5effe1498159116bca960e3ebb196364b WatchSource:0}: Error finding container 7aaa587147681d36b5b21917d898dba5effe1498159116bca960e3ebb196364b: Status 404 returned error can't find the container with id 7aaa587147681d36b5b21917d898dba5effe1498159116bca960e3ebb196364b
Oct 30 22:10:33.743842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.743819    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"]
Oct 30 22:10:33.746621 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:10:33.746590    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62f24f9_d580_474f_a109_ff1c9e879bcf.slice/crio-2325411a825e1631651fc17591b2a2e1d1f019af99579e5650f6d524fd726df7 WatchSource:0}: Error finding container 2325411a825e1631651fc17591b2a2e1d1f019af99579e5650f6d524fd726df7: Status 404 returned error can't find the container with id 2325411a825e1631651fc17591b2a2e1d1f019af99579e5650f6d524fd726df7
Oct 30 22:10:33.845867 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.845826    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66a9c077-71c3-468b-b5fd-12d6ba862263" path="/var/lib/kubelet/pods/66a9c077-71c3-468b-b5fd-12d6ba862263/volumes"
Oct 30 22:10:33.846367 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:33.846331    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9195b46b-8260-4b7a-a879-20c060475ed6" path="/var/lib/kubelet/pods/9195b46b-8260-4b7a-a879-20c060475ed6/volumes"
Oct 30 22:10:34.205127 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.205085    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" event={"ID":"55852795-178a-4281-b268-eff4af7382d2","Type":"ContainerStarted","Data":"dadce0648f65c07bef9539281f3be6c0a03ef70f00e97bc41cb470681322a08a"}
Oct 30 22:10:34.205306 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.205158    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" event={"ID":"55852795-178a-4281-b268-eff4af7382d2","Type":"ContainerStarted","Data":"7aaa587147681d36b5b21917d898dba5effe1498159116bca960e3ebb196364b"}
Oct 30 22:10:34.205306 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.205186    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:34.206924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.206898    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7" event={"ID":"e62f24f9-d580-474f-a109-ff1c9e879bcf","Type":"ContainerStarted","Data":"d15efb0fa9b02a1d26ecb887acfc564fb15ce69a70d5821eba20bc417820e9e0"}
Oct 30 22:10:34.207041 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.206940    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7" event={"ID":"e62f24f9-d580-474f-a109-ff1c9e879bcf","Type":"ContainerStarted","Data":"2325411a825e1631651fc17591b2a2e1d1f019af99579e5650f6d524fd726df7"}
Oct 30 22:10:34.207041 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.206961    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:34.323329 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.323236    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" podStartSLOduration=2.323222721 podStartE2EDuration="2.323222721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:34.321136772 +0000 UTC m=+2182.598830244" watchObservedRunningTime="2026-10-30 22:10:34.323222721 +0000 UTC m=+2182.600916387"
Oct 30 22:10:34.451374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:34.443599    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7" podStartSLOduration=2.443576427 podStartE2EDuration="2.443576427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:34.441529689 +0000 UTC m=+2182.719223173" watchObservedRunningTime="2026-10-30 22:10:34.443576427 +0000 UTC m=+2182.721269899"
Oct 30 22:10:35.475381 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:35.474380    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:10:36.137328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:36.137288    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"]
Oct 30 22:10:36.232241 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:36.232211    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"]
Oct 30 22:10:41.138234 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.138193    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p" podUID="4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" containerName="payload-processing" containerID="cri-o://5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4" gracePeriod=25
Oct 30 22:10:41.237374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.233467    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8" podUID="3788186f-ba6c-447e-babd-a036714b5b17" containerName="payload-pre-processing" containerID="cri-o://63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5" gracePeriod=25
Oct 30 22:10:41.340650 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:41.340581    5561 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3788186f_ba6c_447e_babd_a036714b5b17.slice/crio-63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:10:41.468584 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.468557    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:10:41.524329 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.524301    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-plugins-config-volume\") pod \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") "
Oct 30 22:10:41.524502 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.524471    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-kube-api-access-4czfq\" (UniqueName: \"kubernetes.io/projected/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-kube-api-access-4czfq\") pod \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\" (UID: \"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca\") "
Oct 30 22:10:41.524715 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.524690    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-plugins-config-volume" pod "4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" (UID: "4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:41.524985 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.524962    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:41.526643 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.526617    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-kube-api-access-4czfq" pod "4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" (UID: "4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca"). InnerVolumeSpecName "kube-api-access-4czfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:41.547700 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.547680    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:10:41.625565 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.625532    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3788186f-ba6c-447e-babd-a036714b5b17-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3788186f-ba6c-447e-babd-a036714b5b17-plugins-config-volume\") pod \"3788186f-ba6c-447e-babd-a036714b5b17\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") "
Oct 30 22:10:41.625875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.625854    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3788186f-ba6c-447e-babd-a036714b5b17-plugins-config-volume" pod "3788186f-ba6c-447e-babd-a036714b5b17" (UID: "3788186f-ba6c-447e-babd-a036714b5b17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:41.625875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.625864    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3788186f-ba6c-447e-babd-a036714b5b17-kube-api-access-9fppg\" (UniqueName: \"kubernetes.io/projected/3788186f-ba6c-447e-babd-a036714b5b17-kube-api-access-9fppg\") pod \"3788186f-ba6c-447e-babd-a036714b5b17\" (UID: \"3788186f-ba6c-447e-babd-a036714b5b17\") "
Oct 30 22:10:41.626465 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.626440    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4czfq\" (UniqueName: \"kubernetes.io/projected/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca-kube-api-access-4czfq\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:41.626549 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.626470    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3788186f-ba6c-447e-babd-a036714b5b17-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:41.628043 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.628019    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3788186f-ba6c-447e-babd-a036714b5b17-kube-api-access-9fppg" pod "3788186f-ba6c-447e-babd-a036714b5b17" (UID: "3788186f-ba6c-447e-babd-a036714b5b17"). InnerVolumeSpecName "kube-api-access-9fppg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:41.728078 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:41.728017    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fppg\" (UniqueName: \"kubernetes.io/projected/3788186f-ba6c-447e-babd-a036714b5b17-kube-api-access-9fppg\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:42.268885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.268850    5561 generic.go:356] "Generic (PLEG): container finished" podID="3788186f-ba6c-447e-babd-a036714b5b17" containerID="63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5" exitCode=0
Oct 30 22:10:42.269313 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.268907    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"
Oct 30 22:10:42.269313 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.268930    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8" event={"ID":"3788186f-ba6c-447e-babd-a036714b5b17","Type":"ContainerDied","Data":"63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5"}
Oct 30 22:10:42.269313 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.268990    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8" event={"ID":"3788186f-ba6c-447e-babd-a036714b5b17","Type":"ContainerDied","Data":"98b98c8768eac5c0aad460d8bf612d773b30b15b68a2788bf81c33be33aae5f2"}
Oct 30 22:10:42.269313 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.269013    5561 scope.go:122] "RemoveContainer" containerID="63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5"
Oct 30 22:10:42.270764 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.270738    5561 generic.go:356] "Generic (PLEG): container finished" podID="4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" containerID="5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4" exitCode=0
Oct 30 22:10:42.270878 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.270777    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"
Oct 30 22:10:42.270878 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.270837    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p" event={"ID":"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca","Type":"ContainerDied","Data":"5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4"}
Oct 30 22:10:42.271005 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.270878    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p" event={"ID":"4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca","Type":"ContainerDied","Data":"503b7d204becfa389fccb4ebe3843613d79a8d81f6b5dc6bfe4ee292ff958a3a"}
Oct 30 22:10:42.279588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.279568    5561 scope.go:122] "RemoveContainer" containerID="63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5"
Oct 30 22:10:42.279834 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:42.279815    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5\": container with ID starting with 63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5 not found: ID does not exist" containerID="63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5"
Oct 30 22:10:42.279886 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.279841    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5"} err="failed to get container status \"63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5\": rpc error: code = NotFound desc = could not find container \"63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5\": container with ID starting with 63e9b0b6df7e4daedf8a8439e232715401ed14792029cec2f02c011e7d8094c5 not found: ID does not exist"
Oct 30 22:10:42.279886 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.279857    5561 scope.go:122] "RemoveContainer" containerID="5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4"
Oct 30 22:10:42.299202 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.299184    5561 scope.go:122] "RemoveContainer" containerID="5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4"
Oct 30 22:10:42.299526 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:42.299504    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4\": container with ID starting with 5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4 not found: ID does not exist" containerID="5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4"
Oct 30 22:10:42.299601 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.299532    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4"} err="failed to get container status \"5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4\": rpc error: code = NotFound desc = could not find container \"5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4\": container with ID starting with 5d248c68393ff2177336502f58b84621ee741b93a858b6ef2e5a62404dffd2d4 not found: ID does not exist"
Oct 30 22:10:42.436971 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.436910    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"]
Oct 30 22:10:42.487431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.487409    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8"]
Oct 30 22:10:42.553188 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.553159    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"]
Oct 30 22:10:42.557430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:42.557402    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p"]
Oct 30 22:10:43.845849 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:43.845815    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3788186f-ba6c-447e-babd-a036714b5b17" path="/var/lib/kubelet/pods/3788186f-ba6c-447e-babd-a036714b5b17/volumes"
Oct 30 22:10:43.846219 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:43.846204    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" path="/var/lib/kubelet/pods/4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca/volumes"
Oct 30 22:10:45.217656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:45.217626    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:10:45.218149 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:10:45.217709    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:10:52.981571 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:10:52.981530    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:11:12.844037 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:12.843997    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"]
Oct 30 22:11:21.726436 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:21.726400    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"]
Oct 30 22:11:21.847620 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:21.847589    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:11:22.982677 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:11:22.982639    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:11:27.864476 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:27.864439    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main" containerID="cri-o://3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a" gracePeriod=45
Oct 30 22:11:28.138741 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.138715    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 22:11:28.217182 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217152    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8317276a-8423-494a-8fec-49eb1d43ac3c-tls-certs\" (UniqueName: \"kubernetes.io/secret/8317276a-8423-494a-8fec-49eb1d43ac3c-tls-certs\") pod \"8317276a-8423-494a-8fec-49eb1d43ac3c\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") "
Oct 30 22:11:28.217327 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217259    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-model-cache\") pod \"8317276a-8423-494a-8fec-49eb1d43ac3c\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") "
Oct 30 22:11:28.217418 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217382    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8317276a-8423-494a-8fec-49eb1d43ac3c-kube-api-access-9xdcp\" (UniqueName: \"kubernetes.io/projected/8317276a-8423-494a-8fec-49eb1d43ac3c-kube-api-access-9xdcp\") pod \"8317276a-8423-494a-8fec-49eb1d43ac3c\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") "
Oct 30 22:11:28.217485 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217416    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-dshm\") pod \"8317276a-8423-494a-8fec-49eb1d43ac3c\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") "
Oct 30 22:11:28.217545 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217491    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-home\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-home\") pod \"8317276a-8423-494a-8fec-49eb1d43ac3c\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") "
Oct 30 22:11:28.217545 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217518    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-model-cache" pod "8317276a-8423-494a-8fec-49eb1d43ac3c" (UID: "8317276a-8423-494a-8fec-49eb1d43ac3c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.217545 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217539    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-tmp-dir\") pod \"8317276a-8423-494a-8fec-49eb1d43ac3c\" (UID: \"8317276a-8423-494a-8fec-49eb1d43ac3c\") "
Oct 30 22:11:28.217771 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217728    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-home" pod "8317276a-8423-494a-8fec-49eb1d43ac3c" (UID: "8317276a-8423-494a-8fec-49eb1d43ac3c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.217917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.217890    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-tmp-dir" pod "8317276a-8423-494a-8fec-49eb1d43ac3c" (UID: "8317276a-8423-494a-8fec-49eb1d43ac3c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.218032 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.218018    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.218075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.218039    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.218075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.218052    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.219917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.219888    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-dshm" pod "8317276a-8423-494a-8fec-49eb1d43ac3c" (UID: "8317276a-8423-494a-8fec-49eb1d43ac3c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.220008 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.219963    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8317276a-8423-494a-8fec-49eb1d43ac3c-tls-certs" pod "8317276a-8423-494a-8fec-49eb1d43ac3c" (UID: "8317276a-8423-494a-8fec-49eb1d43ac3c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:28.220049 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.220006    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8317276a-8423-494a-8fec-49eb1d43ac3c-kube-api-access-9xdcp" pod "8317276a-8423-494a-8fec-49eb1d43ac3c" (UID: "8317276a-8423-494a-8fec-49eb1d43ac3c"). InnerVolumeSpecName "kube-api-access-9xdcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:28.319553 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.319522    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8317276a-8423-494a-8fec-49eb1d43ac3c-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.319553 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.319551    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xdcp\" (UniqueName: \"kubernetes.io/projected/8317276a-8423-494a-8fec-49eb1d43ac3c-kube-api-access-9xdcp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.319553 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.319562    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8317276a-8423-494a-8fec-49eb1d43ac3c-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.761022 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.760988    5561 generic.go:356] "Generic (PLEG): container finished" podID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerID="3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a" exitCode=0
Oct 30 22:11:28.761205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.761034    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"
Oct 30 22:11:28.761205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.761029    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" event={"ID":"8317276a-8423-494a-8fec-49eb1d43ac3c","Type":"ContainerDied","Data":"3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a"}
Oct 30 22:11:28.761205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.761146    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq" event={"ID":"8317276a-8423-494a-8fec-49eb1d43ac3c","Type":"ContainerDied","Data":"327938a30d704f5e6add1fb579f198fcf36baf839d020653230bb2ad5365222f"}
Oct 30 22:11:28.761205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.761163    5561 scope.go:122] "RemoveContainer" containerID="3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a"
Oct 30 22:11:28.771184 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.771162    5561 scope.go:122] "RemoveContainer" containerID="3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a"
Oct 30 22:11:28.771540 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:11:28.771520    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a\": container with ID starting with 3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a not found: ID does not exist" containerID="3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a"
Oct 30 22:11:28.771640 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.771545    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a"} err="failed to get container status \"3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a\": rpc error: code = NotFound desc = could not find container \"3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a\": container with ID starting with 3796542f1f43a0936a57881e12117db8ac33c7eb218bc8ba5a0ad71e7d87193a not found: ID does not exist"
Oct 30 22:11:28.927602 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.927568    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"]
Oct 30 22:11:28.933803 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:28.933775    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/simulator-w0-dda9f5-kserve-77f5db79db-jzqpq"]
Oct 30 22:11:29.848887 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:29.848842    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" path="/var/lib/kubelet/pods/8317276a-8423-494a-8fec-49eb1d43ac3c/volumes"
Oct 30 22:11:36.810068 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:36.810009    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main" containerID="cri-o://40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c" gracePeriod=45
Oct 30 22:11:37.078176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.078109    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 22:11:37.161489 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.161455    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6bed758-9227-46c9-ba76-48b1ca055dae-tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bed758-9227-46c9-ba76-48b1ca055dae-tls-certs\") pod \"f6bed758-9227-46c9-ba76-48b1ca055dae\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") "
Oct 30 22:11:37.161696 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.161552    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-dshm\") pod \"f6bed758-9227-46c9-ba76-48b1ca055dae\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") "
Oct 30 22:11:37.161696 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.161638    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-home\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-home\") pod \"f6bed758-9227-46c9-ba76-48b1ca055dae\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") "
Oct 30 22:11:37.161818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.161732    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-model-cache\") pod \"f6bed758-9227-46c9-ba76-48b1ca055dae\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") "
Oct 30 22:11:37.161879 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.161818    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-tmp-dir\") pod \"f6bed758-9227-46c9-ba76-48b1ca055dae\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") "
Oct 30 22:11:37.161879 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.161862    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6bed758-9227-46c9-ba76-48b1ca055dae-kube-api-access-2pzg5\" (UniqueName: \"kubernetes.io/projected/f6bed758-9227-46c9-ba76-48b1ca055dae-kube-api-access-2pzg5\") pod \"f6bed758-9227-46c9-ba76-48b1ca055dae\" (UID: \"f6bed758-9227-46c9-ba76-48b1ca055dae\") "
Oct 30 22:11:37.165391 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.165023    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-home" pod "f6bed758-9227-46c9-ba76-48b1ca055dae" (UID: "f6bed758-9227-46c9-ba76-48b1ca055dae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.167629 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.167584    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-tmp-dir" pod "f6bed758-9227-46c9-ba76-48b1ca055dae" (UID: "f6bed758-9227-46c9-ba76-48b1ca055dae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.167808 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.167776    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-model-cache" pod "f6bed758-9227-46c9-ba76-48b1ca055dae" (UID: "f6bed758-9227-46c9-ba76-48b1ca055dae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.187833 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.187770    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-dshm" pod "f6bed758-9227-46c9-ba76-48b1ca055dae" (UID: "f6bed758-9227-46c9-ba76-48b1ca055dae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.190994 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.190944    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bed758-9227-46c9-ba76-48b1ca055dae-kube-api-access-2pzg5" pod "f6bed758-9227-46c9-ba76-48b1ca055dae" (UID: "f6bed758-9227-46c9-ba76-48b1ca055dae"). InnerVolumeSpecName "kube-api-access-2pzg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:37.191583 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.191538    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6bed758-9227-46c9-ba76-48b1ca055dae-tls-certs" pod "f6bed758-9227-46c9-ba76-48b1ca055dae" (UID: "f6bed758-9227-46c9-ba76-48b1ca055dae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:37.263283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.263252    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.263283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.263280    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.263283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.263290    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.263532 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.263299    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pzg5\" (UniqueName: \"kubernetes.io/projected/f6bed758-9227-46c9-ba76-48b1ca055dae-kube-api-access-2pzg5\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.263532 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.263309    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bed758-9227-46c9-ba76-48b1ca055dae-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.263532 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.263317    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bed758-9227-46c9-ba76-48b1ca055dae-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.807142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.807099    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"]
Oct 30 22:11:37.825108 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.825073    5561 generic.go:356] "Generic (PLEG): container finished" podID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerID="40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c" exitCode=0
Oct 30 22:11:37.825514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.825115    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"
Oct 30 22:11:37.825514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.825160    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" event={"ID":"f6bed758-9227-46c9-ba76-48b1ca055dae","Type":"ContainerDied","Data":"40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c"}
Oct 30 22:11:37.825514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.825205    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8" event={"ID":"f6bed758-9227-46c9-ba76-48b1ca055dae","Type":"ContainerDied","Data":"faff16ff4293c4d4b5557fa01ff43a335f6cac65183a0ec926655d00256ed813"}
Oct 30 22:11:37.825514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.825223    5561 scope.go:122] "RemoveContainer" containerID="40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c"
Oct 30 22:11:37.835098 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.835079    5561 scope.go:122] "RemoveContainer" containerID="40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c"
Oct 30 22:11:37.835397 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:11:37.835377    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c\": container with ID starting with 40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c not found: ID does not exist" containerID="40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c"
Oct 30 22:11:37.835457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:37.835405    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c"} err="failed to get container status \"40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c\": rpc error: code = NotFound desc = could not find container \"40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c\": container with ID starting with 40fa5ef2a6c1d67e51e0d748810dc6964600e2e3861d55fb23b2fc4dad4c734c not found: ID does not exist"
Oct 30 22:11:38.035770 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:38.035737    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"]
Oct 30 22:11:38.038688 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:38.038664    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/premium-w5-9b51e1-kserve-5b5d7b8968-5vgn8"]
Oct 30 22:11:38.274609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:38.274526    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"]
Oct 30 22:11:38.275091 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:38.275054    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" containerID="cri-o://20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53" gracePeriod=30
Oct 30 22:11:39.514467 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:39.514381    5561 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:11:39.514467 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:39.514457    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:11:39.845623 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:39.845550    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" path="/var/lib/kubelet/pods/f6bed758-9227-46c9-ba76-48b1ca055dae/volumes"
Oct 30 22:11:42.807950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:42.807912    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" podUID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" containerName="payload-pre-processing" containerID="cri-o://64cee0cd8d3cabe36bab20145819958fb3978a8f7b8cd7be9a80d6c3ee7ef291" gracePeriod=25
Oct 30 22:11:43.558154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.558095    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 22:11:43.648222 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648193    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b5a84d78-0397-4563-8b25-0d979527427b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a84d78-0397-4563-8b25-0d979527427b-istio-podinfo\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648275    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-envoy\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648444 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648427    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-socket\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648495    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-data\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648598 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648583    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-certs\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648680 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648617    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-credential-socket\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648668    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-socket" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:43.648753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648695    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-istio-token\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-istio-token\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648735    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5a84d78-0397-4563-8b25-0d979527427b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a84d78-0397-4563-8b25-0d979527427b-istiod-ca-cert\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648767    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-kube-api-access-6hdf6\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-kube-api-access-6hdf6\") pod \"b5a84d78-0397-4563-8b25-0d979527427b\" (UID: \"b5a84d78-0397-4563-8b25-0d979527427b\") "
Oct 30 22:11:43.648917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648768    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-data" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:43.648917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648867    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-certs" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:43.648917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.648899    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-credential-socket" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:43.649252 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.649229    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a84d78-0397-4563-8b25-0d979527427b-istiod-ca-cert" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:43.649424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.649396    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.649424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.649425    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.649612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.649440    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.649612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.649454    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.649612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.649467    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a84d78-0397-4563-8b25-0d979527427b-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.650522 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.650503    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-envoy" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:43.650619 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.650518    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5a84d78-0397-4563-8b25-0d979527427b-istio-podinfo" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:43.650808 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.650787    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-kube-api-access-6hdf6" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "kube-api-access-6hdf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:43.650915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.650899    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-istio-token" pod "b5a84d78-0397-4563-8b25-0d979527427b" (UID: "b5a84d78-0397-4563-8b25-0d979527427b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:43.750398 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.750368    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.750398 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.750394    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hdf6\" (UniqueName: \"kubernetes.io/projected/b5a84d78-0397-4563-8b25-0d979527427b-kube-api-access-6hdf6\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.750398 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.750404    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a84d78-0397-4563-8b25-0d979527427b-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.750706 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.750413    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a84d78-0397-4563-8b25-0d979527427b-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:43.862438 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.862405    5561 generic.go:356] "Generic (PLEG): container finished" podID="b5a84d78-0397-4563-8b25-0d979527427b" containerID="20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53" exitCode=0
Oct 30 22:11:43.862854 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.862456    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"
Oct 30 22:11:43.862854 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.862468    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" event={"ID":"b5a84d78-0397-4563-8b25-0d979527427b","Type":"ContainerDied","Data":"20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53"}
Oct 30 22:11:43.862854 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.862508    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh" event={"ID":"b5a84d78-0397-4563-8b25-0d979527427b","Type":"ContainerDied","Data":"c3f88d42cc145dde5277cce67bc128fe4a6034eb6bb971f985f2f5826f91a042"}
Oct 30 22:11:43.862854 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.862528    5561 scope.go:122] "RemoveContainer" containerID="20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53"
Oct 30 22:11:43.872330 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.872313    5561 scope.go:122] "RemoveContainer" containerID="20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53"
Oct 30 22:11:43.872648 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:11:43.872625    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53\": container with ID starting with 20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53 not found: ID does not exist" containerID="20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53"
Oct 30 22:11:43.872752 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.872653    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53"} err="failed to get container status \"20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53\": rpc error: code = NotFound desc = could not find container \"20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53\": container with ID starting with 20d3b1fd650cf60c2bca80f4af7594aa0a477187a2f957735b8c685035fcef53 not found: ID does not exist"
Oct 30 22:11:43.883224 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:43.883199    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" podUID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:11:44.028846 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:44.028816    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"]
Oct 30 22:11:44.034421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:44.034399    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh"]
Oct 30 22:11:45.846149 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:45.846117    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5a84d78-0397-4563-8b25-0d979527427b" path="/var/lib/kubelet/pods/b5a84d78-0397-4563-8b25-0d979527427b/volumes"
Oct 30 22:11:52.986932 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:11:52.986850    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:11:53.883550 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:11:53.883509    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" podUID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:12:02.995119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:02.995082    5561 generic.go:356] "Generic (PLEG): container finished" podID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" containerID="64cee0cd8d3cabe36bab20145819958fb3978a8f7b8cd7be9a80d6c3ee7ef291" exitCode=0
Oct 30 22:12:02.995524 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:02.995216    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" event={"ID":"99b01064-b927-4e4b-9cc9-c5d7dd1d6738","Type":"ContainerDied","Data":"64cee0cd8d3cabe36bab20145819958fb3978a8f7b8cd7be9a80d6c3ee7ef291"}
Oct 30 22:12:03.090187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:03.090157    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 22:12:03.120299 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:03.120267    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-kube-api-access-bphsx\" (UniqueName: \"kubernetes.io/projected/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-kube-api-access-bphsx\") pod \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") "
Oct 30 22:12:03.120480 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:03.120329    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-plugins-config-volume\") pod \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\" (UID: \"99b01064-b927-4e4b-9cc9-c5d7dd1d6738\") "
Oct 30 22:12:03.120730 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:03.120702    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-plugins-config-volume" pod "99b01064-b927-4e4b-9cc9-c5d7dd1d6738" (UID: "99b01064-b927-4e4b-9cc9-c5d7dd1d6738"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:03.120842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:03.120825    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:03.122503 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:03.122484    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-kube-api-access-bphsx" pod "99b01064-b927-4e4b-9cc9-c5d7dd1d6738" (UID: "99b01064-b927-4e4b-9cc9-c5d7dd1d6738"). InnerVolumeSpecName "kube-api-access-bphsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:03.222440 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:03.222406    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bphsx\" (UniqueName: \"kubernetes.io/projected/99b01064-b927-4e4b-9cc9-c5d7dd1d6738-kube-api-access-bphsx\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:04.002848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:04.002823    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"
Oct 30 22:12:04.003302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:04.002849    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4" event={"ID":"99b01064-b927-4e4b-9cc9-c5d7dd1d6738","Type":"ContainerDied","Data":"c4f3525ac04959115034494e57dd5c1b622bea64c2a6914e8f6b65beb1a06fbd"}
Oct 30 22:12:04.003302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:04.002911    5561 scope.go:122] "RemoveContainer" containerID="64cee0cd8d3cabe36bab20145819958fb3978a8f7b8cd7be9a80d6c3ee7ef291"
Oct 30 22:12:04.168988 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:04.168957    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"]
Oct 30 22:12:04.171157 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:04.171134    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4"]
Oct 30 22:12:05.434703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:05.434666    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:12:05.845438 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:05.845332    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" path="/var/lib/kubelet/pods/99b01064-b927-4e4b-9cc9-c5d7dd1d6738/volumes"
Oct 30 22:12:22.981851 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:12:22.981809    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:12:47.146513 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.146477    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"]
Oct 30 22:12:47.147114 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147087    5561 cpu_manager.go:414] "containerMap: removing container" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main"
Oct 30 22:12:47.147114 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147112    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147129    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147138    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147165    5561 cpu_manager.go:414] "containerMap: removing container" podUID="3788186f-ba6c-447e-babd-a036714b5b17" containerName="payload-pre-processing"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147173    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3788186f-ba6c-447e-babd-a036714b5b17" containerName="payload-pre-processing"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147185    5561 cpu_manager.go:414] "containerMap: removing container" podUID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" containerName="payload-pre-processing"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147192    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" containerName="payload-pre-processing"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147201    5561 cpu_manager.go:414] "containerMap: removing container" podUID="4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" containerName="payload-processing"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147206    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" containerName="payload-processing"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147226    5561 cpu_manager.go:414] "containerMap: removing container" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy"
Oct 30 22:12:47.147322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147233    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy"
Oct 30 22:12:47.147689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147392    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5a84d78-0397-4563-8b25-0d979527427b" containerName="istio-proxy"
Oct 30 22:12:47.147689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147408    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6bed758-9227-46c9-ba76-48b1ca055dae" containerName="main"
Oct 30 22:12:47.147689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147415    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8317276a-8423-494a-8fec-49eb1d43ac3c" containerName="main"
Oct 30 22:12:47.147689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147425    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="3788186f-ba6c-447e-babd-a036714b5b17" containerName="payload-pre-processing"
Oct 30 22:12:47.147689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147432    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="99b01064-b927-4e4b-9cc9-c5d7dd1d6738" containerName="payload-pre-processing"
Oct 30 22:12:47.147689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.147439    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c3c27d6-7a97-4350-9e5f-a5d2c7aaafca" containerName="payload-processing"
Oct 30 22:12:47.150741 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.150724    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.153784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.153751    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvrfd\" (UniqueName: \"kubernetes.io/projected/fdf44047-bb4f-4860-a861-a2c5bfd4358b-kube-api-access-dvrfd\") pod \"payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.153890 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.153859    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf44047-bb4f-4860-a861-a2c5bfd4358b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.275912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.275869    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-954628\""
Oct 30 22:12:47.308430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.279608    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvrfd\" (UniqueName: \"kubernetes.io/projected/fdf44047-bb4f-4860-a861-a2c5bfd4358b-kube-api-access-dvrfd\") pod \"payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.308793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.308760    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf44047-bb4f-4860-a861-a2c5bfd4358b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.310509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.286993    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"]
Oct 30 22:12:47.310723 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.310704    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"]
Oct 30 22:12:47.310947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.309779    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf44047-bb4f-4860-a861-a2c5bfd4358b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.310947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.282642    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-954628-dockercfg-sfmgz\""
Oct 30 22:12:47.336230 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.336187    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.350557 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.350533    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvrfd\" (UniqueName: \"kubernetes.io/projected/fdf44047-bb4f-4860-a861-a2c5bfd4358b-kube-api-access-dvrfd\") pod \"payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.375718 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.375692    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"]
Oct 30 22:12:47.412076 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.412001    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nlvp\" (UniqueName: \"kubernetes.io/projected/6a0c8114-242f-4cdd-9a38-eff32ed82211-kube-api-access-4nlvp\") pod \"payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") " pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.412284 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.412262    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a0c8114-242f-4cdd-9a38-eff32ed82211-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") " pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.465040 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.465012    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:47.513768 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.513735    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a0c8114-242f-4cdd-9a38-eff32ed82211-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") " pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.513930 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.513845    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nlvp\" (UniqueName: \"kubernetes.io/projected/6a0c8114-242f-4cdd-9a38-eff32ed82211-kube-api-access-4nlvp\") pod \"payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") " pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.514471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.514447    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a0c8114-242f-4cdd-9a38-eff32ed82211-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") " pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.585505 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.585473    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nlvp\" (UniqueName: \"kubernetes.io/projected/6a0c8114-242f-4cdd-9a38-eff32ed82211-kube-api-access-4nlvp\") pod \"payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") " pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.653985 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.653958    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:47.902094 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:47.900569    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"]
Oct 30 22:12:48.045915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.045887    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"]
Oct 30 22:12:48.046476 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:12:48.046453    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0c8114_242f_4cdd_9a38_eff32ed82211.slice/crio-e555596cfaed5c09064f04f9ac6f4ef2db5fabab463fdabcbe4749fde0552cda WatchSource:0}: Error finding container e555596cfaed5c09064f04f9ac6f4ef2db5fabab463fdabcbe4749fde0552cda: Status 404 returned error can't find the container with id e555596cfaed5c09064f04f9ac6f4ef2db5fabab463fdabcbe4749fde0552cda
Oct 30 22:12:48.350597 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.350512    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" event={"ID":"fdf44047-bb4f-4860-a861-a2c5bfd4358b","Type":"ContainerStarted","Data":"3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2"}
Oct 30 22:12:48.350597 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.350568    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" event={"ID":"fdf44047-bb4f-4860-a861-a2c5bfd4358b","Type":"ContainerStarted","Data":"4f495b55e722bc56c0b8afda6a61b42e8ee9ef3b1afe190a57efb0780a214d25"}
Oct 30 22:12:48.351026 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.350622    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:12:48.352386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.352342    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c" event={"ID":"6a0c8114-242f-4cdd-9a38-eff32ed82211","Type":"ContainerStarted","Data":"2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692"}
Oct 30 22:12:48.352509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.352404    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c" event={"ID":"6a0c8114-242f-4cdd-9a38-eff32ed82211","Type":"ContainerStarted","Data":"e555596cfaed5c09064f04f9ac6f4ef2db5fabab463fdabcbe4749fde0552cda"}
Oct 30 22:12:48.352509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.352474    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:48.468475 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.468431    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" podStartSLOduration=1.46841906 podStartE2EDuration="1.46841906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:48.466621751 +0000 UTC m=+2316.744315244" watchObservedRunningTime="2026-10-30 22:12:48.46841906 +0000 UTC m=+2316.746112559"
Oct 30 22:12:48.584398 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:48.584316    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c" podStartSLOduration=1.584297858 podStartE2EDuration="1.584297858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:48.583175788 +0000 UTC m=+2316.860869260" watchObservedRunningTime="2026-10-30 22:12:48.584297858 +0000 UTC m=+2316.861991330"
Oct 30 22:12:52.981469 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:12:52.981428    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:12:55.777531 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.777495    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"]
Oct 30 22:12:55.781752 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.781727    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.831602 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.831565    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.831821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.831650    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.831821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.831761    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.831821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.831811    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvph7\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-kube-api-access-nvph7\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.832010 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.831986    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.832071 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.832035    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.832126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.832098    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.832179 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.832143    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.832231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.832199    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.837541 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.835653    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6df3ba0e-openshift-default-dockercfg-6ckx9\""
Oct 30 22:12:55.888517 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.888486    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"]
Oct 30 22:12:55.933174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933141    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933217    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933252    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933284    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvph7\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-kube-api-access-nvph7\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933383    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933418    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933479    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933513    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933552    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933657    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933692    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.933932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933909    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.934038 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933932    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.934038 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.933991    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.935748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.935721    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.935877 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.935860    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.991755 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.991729    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:55.991921 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:55.991898    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvph7\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-kube-api-access-nvph7\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:56.063734 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:56.063652    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"]
Oct 30 22:12:56.063915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:56.063886    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy" containerID="cri-o://6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b" gracePeriod=30
Oct 30 22:12:56.097635 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:56.097597    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:56.582282 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:56.582250    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"]
Oct 30 22:12:56.583552 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:12:56.583529    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77dc14e6_0063_4b6e_95fc_1be5490e2f5e.slice/crio-88c3b0ab77d7059d6c385f7269d35c74ee99a1ac1908202b19a883eb9bb72bf0 WatchSource:0}: Error finding container 88c3b0ab77d7059d6c385f7269d35c74ee99a1ac1908202b19a883eb9bb72bf0: Status 404 returned error can't find the container with id 88c3b0ab77d7059d6c385f7269d35c74ee99a1ac1908202b19a883eb9bb72bf0
Oct 30 22:12:56.586464 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:56.586400    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:56.586548 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:56.586494    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:56.586548 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:56.586541    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:12:57.411570 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:57.411531    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" event={"ID":"77dc14e6-0063-4b6e-95fc-1be5490e2f5e","Type":"ContainerStarted","Data":"6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9"}
Oct 30 22:12:57.411971 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:57.411597    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" event={"ID":"77dc14e6-0063-4b6e-95fc-1be5490e2f5e","Type":"ContainerStarted","Data":"88c3b0ab77d7059d6c385f7269d35c74ee99a1ac1908202b19a883eb9bb72bf0"}
Oct 30 22:12:57.641446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:57.641394    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podStartSLOduration=2.6413808640000003 podStartE2EDuration="2.641380864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:57.588842137 +0000 UTC m=+2325.866535613" watchObservedRunningTime="2026-10-30 22:12:57.641380864 +0000 UTC m=+2325.919074402"
Oct 30 22:12:58.098105 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:58.098075    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:12:58.099766 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:58.099738    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:12:58.099899 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:58.099785    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:12:59.098111 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:59.098079    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:12:59.098527 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:59.098127    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:12:59.369490 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:59.369411    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:12:59.370232 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:12:59.370209    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:13:00.098589 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:00.098553    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:13:00.099083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:00.098631    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:13:01.098734 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.098704    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:13:01.099132 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.098754    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:13:01.378081 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.378055    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 22:13:01.437240 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.437205    5561 generic.go:356] "Generic (PLEG): container finished" podID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerID="6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b" exitCode=0
Oct 30 22:13:01.437405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.437255    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"
Oct 30 22:13:01.437405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.437295    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" event={"ID":"e1c0dffe-c267-4356-aa24-03b78c24dd49","Type":"ContainerDied","Data":"6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b"}
Oct 30 22:13:01.437405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.437382    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc" event={"ID":"e1c0dffe-c267-4356-aa24-03b78c24dd49","Type":"ContainerDied","Data":"cab4e85ca09b8696de504b47beaea50c861db83a9fcc855976108c9841959637"}
Oct 30 22:13:01.437548 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.437413    5561 scope.go:122] "RemoveContainer" containerID="6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b"
Oct 30 22:13:01.475656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.475626    5561 scope.go:122] "RemoveContainer" containerID="b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4"
Oct 30 22:13:01.485929 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.485911    5561 scope.go:122] "RemoveContainer" containerID="6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b"
Oct 30 22:13:01.486200 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:13:01.486175    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b\": container with ID starting with 6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b not found: ID does not exist" containerID="6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b"
Oct 30 22:13:01.486265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.486210    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b"} err="failed to get container status \"6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b\": rpc error: code = NotFound desc = could not find container \"6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b\": container with ID starting with 6885aef31d9508bcd28ff3ebea8b850f756bd5a5105313be133ee9bd0d92091b not found: ID does not exist"
Oct 30 22:13:01.486265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.486228    5561 scope.go:122] "RemoveContainer" containerID="b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4"
Oct 30 22:13:01.486510 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:13:01.486493    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4\": container with ID starting with b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4 not found: ID does not exist" containerID="b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4"
Oct 30 22:13:01.486588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.486519    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4"} err="failed to get container status \"b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4\": rpc error: code = NotFound desc = could not find container \"b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4\": container with ID starting with b58d9d86632de350350bca0e003fae47133bbf0cc03b8568599ad787c67a77f4 not found: ID does not exist"
Oct 30 22:13:01.540757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.540713    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-certs\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.540909 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.540780    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-data\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.540909 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.540871    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-token\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-token\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.541032 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.540935    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-socket\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.541032 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.540995    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-credential-socket\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.541032 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541019    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-certs" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:01.541191 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541049    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-data" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:01.541191 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541051    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1c0dffe-c267-4356-aa24-03b78c24dd49-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c0dffe-c267-4356-aa24-03b78c24dd49-istiod-ca-cert\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.541191 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541120    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-kube-api-access-4qfgm\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-kube-api-access-4qfgm\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.541394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541249    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-podinfo\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.541394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541286    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-envoy\") pod \"e1c0dffe-c267-4356-aa24-03b78c24dd49\" (UID: \"e1c0dffe-c267-4356-aa24-03b78c24dd49\") "
Oct 30 22:13:01.541940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541254    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-socket" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:01.541940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541413    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c0dffe-c267-4356-aa24-03b78c24dd49-istiod-ca-cert" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:01.541940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541664    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-credential-socket" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:01.541940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541846    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.541940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.541903    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.569415 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.559515    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-podinfo" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:13:01.572117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.571964    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-token" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:01.578764 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.573468    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-kube-api-access-4qfgm" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "kube-api-access-4qfgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:01.578764 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.575324    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-envoy" pod "e1c0dffe-c267-4356-aa24-03b78c24dd49" (UID: "e1c0dffe-c267-4356-aa24-03b78c24dd49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:01.668379 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.643859    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.668608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.668490    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.668608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.668526    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.668608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.668541    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.668608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.668556    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c0dffe-c267-4356-aa24-03b78c24dd49-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.668608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.668571    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c0dffe-c267-4356-aa24-03b78c24dd49-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.668608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.668585    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qfgm\" (UniqueName: \"kubernetes.io/projected/e1c0dffe-c267-4356-aa24-03b78c24dd49-kube-api-access-4qfgm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:01.886184 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.886148    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"]
Oct 30 22:13:01.889942 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:01.889913    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc"]
Oct 30 22:13:02.097934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:02.097856    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:13:02.097934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:02.097905    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:13:03.461299 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:03.461265    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:13:03.462231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:03.462206    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:13:03.462966 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:03.462935    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:13:03.846509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:03.846438    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" path="/var/lib/kubelet/pods/e1c0dffe-c267-4356-aa24-03b78c24dd49/volumes"
Oct 30 22:13:18.660259 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:18.660221    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:13:22.981150 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:13:22.981051    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:13:26.029365 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.029329    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"]
Oct 30 22:13:26.030142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.030120    5561 cpu_manager.go:414] "containerMap: removing container" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy"
Oct 30 22:13:26.030142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.030142    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy"
Oct 30 22:13:26.030304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.030166    5561 cpu_manager.go:414] "containerMap: removing container" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy"
Oct 30 22:13:26.030304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.030173    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy"
Oct 30 22:13:26.030438 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.030330    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy"
Oct 30 22:13:26.030438 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.030347    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1c0dffe-c267-4356-aa24-03b78c24dd49" containerName="istio-proxy"
Oct 30 22:13:26.034953 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.034933    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.088946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.088918    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.088964    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.089066    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.089086    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9457db31-c6aa-421f-a256-4f74be49b9e0-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.089108    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6cxt\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-kube-api-access-m6cxt\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089308 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.089215    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089308 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.089276    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.089336    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.089430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.089374    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.140824 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.140794    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"]
Oct 30 22:13:26.190231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190200    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190418 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190271    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190487 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190412    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190487 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190454    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9457db31-c6aa-421f-a256-4f74be49b9e0-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190596 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190491    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cxt\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-kube-api-access-m6cxt\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190596 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190560    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190627    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190674    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190812 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190710    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.190812 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.190723    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.191135 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.191112    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.191287 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.191142    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9457db31-c6aa-421f-a256-4f74be49b9e0-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.191418 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.191174    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.191418 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.191188    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.197382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.193936    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.197382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.195977    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.252183 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.252160    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.269809 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.269775    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6cxt\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-kube-api-access-m6cxt\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.348231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.348172    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:26.773650 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.773620    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"]
Oct 30 22:13:26.774955 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:13:26.774925    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9457db31_c6aa_421f_a256_4f74be49b9e0.slice/crio-d7e3d6f275fcc5df01074d0e3f0427e74a3b792c904ec31b9b245dfda5676844 WatchSource:0}: Error finding container d7e3d6f275fcc5df01074d0e3f0427e74a3b792c904ec31b9b245dfda5676844: Status 404 returned error can't find the container with id d7e3d6f275fcc5df01074d0e3f0427e74a3b792c904ec31b9b245dfda5676844
Oct 30 22:13:26.777420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.777388    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:26.777498 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.777451    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:26.777558 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:26.777496    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:27.676255 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:27.676212    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" event={"ID":"9457db31-c6aa-421f-a256-4f74be49b9e0","Type":"ContainerStarted","Data":"df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de"}
Oct 30 22:13:27.676753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:27.676310    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" event={"ID":"9457db31-c6aa-421f-a256-4f74be49b9e0","Type":"ContainerStarted","Data":"d7e3d6f275fcc5df01074d0e3f0427e74a3b792c904ec31b9b245dfda5676844"}
Oct 30 22:13:28.105527 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.105424    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podStartSLOduration=3.105409802 podStartE2EDuration="3.105409802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:27.864694338 +0000 UTC m=+2356.142387792" watchObservedRunningTime="2026-10-30 22:13:28.105409802 +0000 UTC m=+2356.383103272"
Oct 30 22:13:28.107618 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.107595    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"]
Oct 30 22:13:28.107801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.107782    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck" podUID="d88b0307-6421-42cb-9bf4-8cef89e95af7" containerName="maas-api" containerID="cri-o://233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae" gracePeriod=30
Oct 30 22:13:28.363430 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.350867    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:28.405588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.405535    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:28.405780 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.405609    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:28.477719 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.477688    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 22:13:28.549366 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.533961    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-metrics-certs\") pod \"d88b0307-6421-42cb-9bf4-8cef89e95af7\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") "
Oct 30 22:13:28.549366 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.544447    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d88b0307-6421-42cb-9bf4-8cef89e95af7-kube-api-access-xxwvz\" (UniqueName: \"kubernetes.io/projected/d88b0307-6421-42cb-9bf4-8cef89e95af7-kube-api-access-xxwvz\") pod \"d88b0307-6421-42cb-9bf4-8cef89e95af7\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") "
Oct 30 22:13:28.549366 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.544754    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls\") pod \"d88b0307-6421-42cb-9bf4-8cef89e95af7\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") "
Oct 30 22:13:28.549366 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.544830    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d88b0307-6421-42cb-9bf4-8cef89e95af7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88b0307-6421-42cb-9bf4-8cef89e95af7-openshift-service-ca\") pod \"d88b0307-6421-42cb-9bf4-8cef89e95af7\" (UID: \"d88b0307-6421-42cb-9bf4-8cef89e95af7\") "
Oct 30 22:13:28.570390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.562370    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-metrics-certs" pod "d88b0307-6421-42cb-9bf4-8cef89e95af7" (UID: "d88b0307-6421-42cb-9bf4-8cef89e95af7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:28.570390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.568987    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d88b0307-6421-42cb-9bf4-8cef89e95af7-openshift-service-ca" pod "d88b0307-6421-42cb-9bf4-8cef89e95af7" (UID: "d88b0307-6421-42cb-9bf4-8cef89e95af7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:28.579999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.579962    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls" pod "d88b0307-6421-42cb-9bf4-8cef89e95af7" (UID: "d88b0307-6421-42cb-9bf4-8cef89e95af7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:28.580646 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.580612    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88b0307-6421-42cb-9bf4-8cef89e95af7-kube-api-access-xxwvz" pod "d88b0307-6421-42cb-9bf4-8cef89e95af7" (UID: "d88b0307-6421-42cb-9bf4-8cef89e95af7"). InnerVolumeSpecName "kube-api-access-xxwvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:28.662375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.656558    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:28.662375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.656604    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d88b0307-6421-42cb-9bf4-8cef89e95af7-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:28.662375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.656622    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d88b0307-6421-42cb-9bf4-8cef89e95af7-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:28.662375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.656638    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxwvz\" (UniqueName: \"kubernetes.io/projected/d88b0307-6421-42cb-9bf4-8cef89e95af7-kube-api-access-xxwvz\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:28.710999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.707660    5561 generic.go:356] "Generic (PLEG): container finished" podID="d88b0307-6421-42cb-9bf4-8cef89e95af7" containerID="233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae" exitCode=0
Oct 30 22:13:28.710999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.708056    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"
Oct 30 22:13:28.710999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.708501    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck" event={"ID":"d88b0307-6421-42cb-9bf4-8cef89e95af7","Type":"ContainerDied","Data":"233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae"}
Oct 30 22:13:28.710999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.708567    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck" event={"ID":"d88b0307-6421-42cb-9bf4-8cef89e95af7","Type":"ContainerDied","Data":"b0733204195d0f0236cfe940617678d9b5e7dec1bf8a321e66020b6ac32a3140"}
Oct 30 22:13:28.710999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.708595    5561 scope.go:122] "RemoveContainer" containerID="233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae"
Oct 30 22:13:28.726528 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.723031    5561 scope.go:122] "RemoveContainer" containerID="233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae"
Oct 30 22:13:28.726528 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:13:28.723315    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae\": container with ID starting with 233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae not found: ID does not exist" containerID="233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae"
Oct 30 22:13:28.726528 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.723345    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae"} err="failed to get container status \"233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae\": rpc error: code = NotFound desc = could not find container \"233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae\": container with ID starting with 233bcbf41f96b33ee5fe30d40f906f12c87fb0826f8886be647b2fac0af07aae not found: ID does not exist"
Oct 30 22:13:28.887847 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.887817    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"]
Oct 30 22:13:28.901276 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:28.901246    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-9b51e1-648fb46b59-gtbck"]
Oct 30 22:13:29.350578 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:29.348750    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:29.350578 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:29.348808    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:29.870662 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:29.870617    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d88b0307-6421-42cb-9bf4-8cef89e95af7" path="/var/lib/kubelet/pods/d88b0307-6421-42cb-9bf4-8cef89e95af7/volumes"
Oct 30 22:13:30.349444 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:30.348715    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:30.349444 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:30.348774    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:31.362729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:31.362693    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:31.363337 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:31.363302    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:32.366382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:32.358700    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:32.366382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:32.358762    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:33.353799 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:33.353761    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:33.354173 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:33.354151    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:33.354858 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:33.354837    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:13:38.573037 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:38.572998    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"]
Oct 30 22:13:38.573558 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:38.573263    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy" containerID="cri-o://26238087ddd5c3c3b9749829c09ff4af9dd0dabf05495e40fe1f484b6a6e20f5" gracePeriod=30
Oct 30 22:13:41.139997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.139961    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"]
Oct 30 22:13:41.140636 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.140618    5561 cpu_manager.go:414] "containerMap: removing container" podUID="d88b0307-6421-42cb-9bf4-8cef89e95af7" containerName="maas-api"
Oct 30 22:13:41.140697 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.140638    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d88b0307-6421-42cb-9bf4-8cef89e95af7" containerName="maas-api"
Oct 30 22:13:41.140761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.140751    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="d88b0307-6421-42cb-9bf4-8cef89e95af7" containerName="maas-api"
Oct 30 22:13:41.143988 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.143972    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.266753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.266712    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.266939 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.266801    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.266939 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.266885    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jms6z\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-kube-api-access-jms6z\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.266939 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.266931    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.267114 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.267043    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.267172 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.267122    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.267172 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.267165    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.267322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.267296    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.267467 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.267449    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.365615 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.365581    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"]
Oct 30 22:13:41.368746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.368719    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.368876 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.368762    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.368876 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.368816    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369005 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.368893    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369005 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.368960    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369025    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369067    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jms6z\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-kube-api-access-jms6z\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369230 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369110    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369230 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369167    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369230 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369174    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369565 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369542    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369767 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369744    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.369853 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.369811    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.370249 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.370230    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.371606 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.371581    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.371955 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.371931    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.428532 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.428471    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.428979 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.428960    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jms6z\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-kube-api-access-jms6z\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.457102 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.457083    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:41.934146 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.934116    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"]
Oct 30 22:13:41.934775 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:13:41.934749    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db6bc39_c2df_4ebe_b945_4e2b3ab3d93d.slice/crio-e9f3c95b90908184eb685e5995a7b73f7483c0d40bd868f4996d626beff32d46 WatchSource:0}: Error finding container e9f3c95b90908184eb685e5995a7b73f7483c0d40bd868f4996d626beff32d46: Status 404 returned error can't find the container with id e9f3c95b90908184eb685e5995a7b73f7483c0d40bd868f4996d626beff32d46
Oct 30 22:13:41.937145 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.937111    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:41.937220 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.937172    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:41.937220 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:41.937200    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:42.876378 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:42.876333    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" event={"ID":"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d","Type":"ContainerStarted","Data":"820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0"}
Oct 30 22:13:42.876845 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:42.876398    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" event={"ID":"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d","Type":"ContainerStarted","Data":"e9f3c95b90908184eb685e5995a7b73f7483c0d40bd868f4996d626beff32d46"}
Oct 30 22:13:43.005800 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:43.005746    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podStartSLOduration=2.005731187 podStartE2EDuration="2.005731187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:43.002597528 +0000 UTC m=+2371.280291000" watchObservedRunningTime="2026-10-30 22:13:43.005731187 +0000 UTC m=+2371.283424657"
Oct 30 22:13:43.460094 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:43.459521    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:43.460094 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:43.459669    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 22:13:43.460094 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:43.459726    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 22:13:43.906241 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:43.906210    5561 generic.go:356] "Generic (PLEG): container finished" podID="ec13631d-151c-4fec-b662-34280fc56a06" containerID="26238087ddd5c3c3b9749829c09ff4af9dd0dabf05495e40fe1f484b6a6e20f5" exitCode=0
Oct 30 22:13:43.906736 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:43.906264    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" event={"ID":"ec13631d-151c-4fec-b662-34280fc56a06","Type":"ContainerDied","Data":"26238087ddd5c3c3b9749829c09ff4af9dd0dabf05495e40fe1f484b6a6e20f5"}
Oct 30 22:13:43.926115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:43.926092    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 22:13:44.003530 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003498    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-credential-socket\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.003727 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003575    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec13631d-151c-4fec-b662-34280fc56a06-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec13631d-151c-4fec-b662-34280fc56a06-istiod-ca-cert\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.003727 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003618    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-kube-api-access-8s66z\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-kube-api-access-8s66z\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.003727 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003671    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-certs\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.003727 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003710    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-istio-token\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-istio-token\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.003924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003743    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-data\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.003924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003817    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-credential-socket" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:44.004024 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003920    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-socket\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.004024 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003972    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-envoy\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.004024 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.003973    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-certs" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:44.004173 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.004134    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec13631d-151c-4fec-b662-34280fc56a06-istiod-ca-cert" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:44.004241 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.004191    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-socket" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:44.004296 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.004281    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec13631d-151c-4fec-b662-34280fc56a06-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec13631d-151c-4fec-b662-34280fc56a06-istio-podinfo\") pod \"ec13631d-151c-4fec-b662-34280fc56a06\" (UID: \"ec13631d-151c-4fec-b662-34280fc56a06\") "
Oct 30 22:13:44.004419 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.004394    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-data" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:44.005270 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.005247    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.005463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.005438    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.005579 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.005465    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec13631d-151c-4fec-b662-34280fc56a06-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.005579 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.005482    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.005579 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.005496    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.006475 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.006454    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-envoy" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:44.006574 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.006528    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-kube-api-access-8s66z" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "kube-api-access-8s66z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:44.006781 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.006763    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-istio-token" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:44.007422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.007398    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec13631d-151c-4fec-b662-34280fc56a06-istio-podinfo" pod "ec13631d-151c-4fec-b662-34280fc56a06" (UID: "ec13631d-151c-4fec-b662-34280fc56a06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:13:44.106853 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.106816    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s66z\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-kube-api-access-8s66z\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.106853 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.106853    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec13631d-151c-4fec-b662-34280fc56a06-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.107042 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.106868    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec13631d-151c-4fec-b662-34280fc56a06-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.107042 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.106882    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec13631d-151c-4fec-b662-34280fc56a06-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:44.464373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.463671    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 22:13:44.464373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.463731    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 22:13:44.933667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.932231    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9" event={"ID":"ec13631d-151c-4fec-b662-34280fc56a06","Type":"ContainerDied","Data":"278e27a7980d199bbdedee2cce46a71977b06e2db0ac4632483fc430408af2cb"}
Oct 30 22:13:44.933667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.932313    5561 scope.go:122] "RemoveContainer" containerID="26238087ddd5c3c3b9749829c09ff4af9dd0dabf05495e40fe1f484b6a6e20f5"
Oct 30 22:13:44.933667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:44.932746    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"
Oct 30 22:13:45.103658 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:45.103624    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"]
Oct 30 22:13:45.170482 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:45.170445    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9"]
Oct 30 22:13:45.459746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:45.459707    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 22:13:45.460030 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:45.460002    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 22:13:45.868509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:45.868466    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec13631d-151c-4fec-b662-34280fc56a06" path="/var/lib/kubelet/pods/ec13631d-151c-4fec-b662-34280fc56a06/volumes"
Oct 30 22:13:46.466477 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:46.466440    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 22:13:46.467062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:46.467028    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 22:13:47.458720 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:47.458681    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 22:13:47.458985 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:47.458952    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 22:13:48.461606 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:48.461574    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:48.461984 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:48.461965    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:48.462727 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:48.462701    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:13:50.127641 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:50.127610    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"]
Oct 30 22:13:50.177921 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:50.177890    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"]
Oct 30 22:13:52.980616 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:13:52.980581    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:13:57.264776 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:13:57.264737    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:14:00.722152 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:00.722116    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"]
Oct 30 22:14:00.772912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:00.772884    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"]
Oct 30 22:14:05.149029 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.148988    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main" containerID="cri-o://b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679" gracePeriod=45
Oct 30 22:14:05.197489 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.197451    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main" containerID="cri-o://de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263" gracePeriod=45
Oct 30 22:14:05.477830 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.477809    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 22:14:05.481179 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.481154    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 22:14:05.529086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529062    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-dshm\") pod \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") "
Oct 30 22:14:05.529182 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529123    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b887b392-ad21-4cfa-893e-ee93afcfc3d5-kube-api-access-q5x45\" (UniqueName: \"kubernetes.io/projected/b887b392-ad21-4cfa-893e-ee93afcfc3d5-kube-api-access-q5x45\") pod \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") "
Oct 30 22:14:05.529182 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529159    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-home\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-home\") pod \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") "
Oct 30 22:14:05.529272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529183    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-model-cache\") pod \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") "
Oct 30 22:14:05.529272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529222    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-home\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-home\") pod \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") "
Oct 30 22:14:05.529410 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529286    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tmp-dir\") pod \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") "
Oct 30 22:14:05.529410 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529339    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tmp-dir\") pod \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") "
Oct 30 22:14:05.529528 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529425    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-model-cache\") pod \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") "
Oct 30 22:14:05.529528 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529467    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tls-certs\" (UniqueName: \"kubernetes.io/secret/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tls-certs\") pod \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") "
Oct 30 22:14:05.529528 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529459    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-home" pod "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" (UID: "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.529528 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529497    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tls-certs\" (UniqueName: \"kubernetes.io/secret/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tls-certs\") pod \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") "
Oct 30 22:14:05.529754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529549    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-home" pod "b887b392-ad21-4cfa-893e-ee93afcfc3d5" (UID: "b887b392-ad21-4cfa-893e-ee93afcfc3d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.529754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529532    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-model-cache" pod "b887b392-ad21-4cfa-893e-ee93afcfc3d5" (UID: "b887b392-ad21-4cfa-893e-ee93afcfc3d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.529754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529573    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tmp-dir" pod "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" (UID: "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.529754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529635    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-kube-api-access-9zqcr\" (UniqueName: \"kubernetes.io/projected/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-kube-api-access-9zqcr\") pod \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\" (UID: \"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37\") "
Oct 30 22:14:05.529754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529667    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-model-cache" pod "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" (UID: "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.529754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529749    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-dshm\") pod \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\" (UID: \"b887b392-ad21-4cfa-893e-ee93afcfc3d5\") "
Oct 30 22:14:05.530149 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.529908    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tmp-dir" pod "b887b392-ad21-4cfa-893e-ee93afcfc3d5" (UID: "b887b392-ad21-4cfa-893e-ee93afcfc3d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.530692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.530672    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.530847 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.530829    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.530978 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.530963    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.531105 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.531092    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.531226 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.531204    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.531226 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.531227    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.531819 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.531777    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-dshm" pod "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" (UID: "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.531907 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.531865    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tls-certs" pod "b887b392-ad21-4cfa-893e-ee93afcfc3d5" (UID: "b887b392-ad21-4cfa-893e-ee93afcfc3d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:05.531907 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.531898    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b887b392-ad21-4cfa-893e-ee93afcfc3d5-kube-api-access-q5x45" pod "b887b392-ad21-4cfa-893e-ee93afcfc3d5" (UID: "b887b392-ad21-4cfa-893e-ee93afcfc3d5"). InnerVolumeSpecName "kube-api-access-q5x45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:05.532007 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.531986    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tls-certs" pod "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" (UID: "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:05.532042 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.532026    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-kube-api-access-9zqcr" pod "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" (UID: "b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37"). InnerVolumeSpecName "kube-api-access-9zqcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:05.532176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.532155    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-dshm" pod "b887b392-ad21-4cfa-893e-ee93afcfc3d5" (UID: "b887b392-ad21-4cfa-893e-ee93afcfc3d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:05.632341 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.632309    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zqcr\" (UniqueName: \"kubernetes.io/projected/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-kube-api-access-9zqcr\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.632341 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.632344    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b887b392-ad21-4cfa-893e-ee93afcfc3d5-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.632533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.632380    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.632533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.632393    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5x45\" (UniqueName: \"kubernetes.io/projected/b887b392-ad21-4cfa-893e-ee93afcfc3d5-kube-api-access-q5x45\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.632533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.632406    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:05.632533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:05.632422    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b887b392-ad21-4cfa-893e-ee93afcfc3d5-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:06.091520 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.091480    5561 generic.go:356] "Generic (PLEG): container finished" podID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerID="de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263" exitCode=0
Oct 30 22:14:06.091703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.091524    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"
Oct 30 22:14:06.091703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.091561    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" event={"ID":"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37","Type":"ContainerDied","Data":"de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263"}
Oct 30 22:14:06.091703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.091609    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj" event={"ID":"b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37","Type":"ContainerDied","Data":"73985d5f347030054307c83c17eb82525f943d8406bb3d8608c1ce80a05b99b3"}
Oct 30 22:14:06.091703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.091631    5561 scope.go:122] "RemoveContainer" containerID="de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263"
Oct 30 22:14:06.093106 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.093078    5561 generic.go:356] "Generic (PLEG): container finished" podID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerID="b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679" exitCode=0
Oct 30 22:14:06.093245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.093128    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"
Oct 30 22:14:06.093245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.093173    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" event={"ID":"b887b392-ad21-4cfa-893e-ee93afcfc3d5","Type":"ContainerDied","Data":"b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679"}
Oct 30 22:14:06.093245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.093207    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7" event={"ID":"b887b392-ad21-4cfa-893e-ee93afcfc3d5","Type":"ContainerDied","Data":"cf524374297e1624d1523865007e493815933aabfb2e3e479b08c72f29eecf18"}
Oct 30 22:14:06.105279 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.105259    5561 scope.go:122] "RemoveContainer" containerID="de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263"
Oct 30 22:14:06.105678 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:06.105647    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263\": container with ID starting with de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263 not found: ID does not exist" containerID="de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263"
Oct 30 22:14:06.105769 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.105686    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263"} err="failed to get container status \"de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263\": rpc error: code = NotFound desc = could not find container \"de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263\": container with ID starting with de1fa840b5208ba0eaa340a6e091c9d3b6906a7f54e81d016747e54ff5a36263 not found: ID does not exist"
Oct 30 22:14:06.105769 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.105710    5561 scope.go:122] "RemoveContainer" containerID="b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679"
Oct 30 22:14:06.114530 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.114509    5561 scope.go:122] "RemoveContainer" containerID="b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679"
Oct 30 22:14:06.114785 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:06.114763    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679\": container with ID starting with b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679 not found: ID does not exist" containerID="b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679"
Oct 30 22:14:06.114834 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.114793    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679"} err="failed to get container status \"b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679\": rpc error: code = NotFound desc = could not find container \"b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679\": container with ID starting with b7adb6e7304abd55134bcffadde1cd679887cb3a64b5dfe2901fd09009a2a679 not found: ID does not exist"
Oct 30 22:14:06.258670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.258640    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"]
Oct 30 22:14:06.261197 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.261171    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-2-w4-c3e2d8-kserve-96784d6c9-r27n7"]
Oct 30 22:14:06.418745 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.418717    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"]
Oct 30 22:14:06.422755 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:06.422731    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-c3e2d8/distinct-w4-c3e2d8-kserve-855c9694c6-dvslj"]
Oct 30 22:14:07.847874 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:07.847838    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" path="/var/lib/kubelet/pods/b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37/volumes"
Oct 30 22:14:07.848362 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:07.848334    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" path="/var/lib/kubelet/pods/b887b392-ad21-4cfa-893e-ee93afcfc3d5/volumes"
Oct 30 22:14:12.316716 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:12.316690    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:14:12.362456 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:12.362431    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:14:15.742993 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:15.742949    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main" containerID="cri-o://e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb" gracePeriod=45
Oct 30 22:14:15.792954 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:15.792895    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main" containerID="cri-o://900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81" gracePeriod=45
Oct 30 22:14:16.083862 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.083837    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 22:14:16.087198 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.087177    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 22:14:16.165595 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165572    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-tmp-dir\") pod \"a9667903-bf26-4c66-b4cf-fecb659a03f2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") "
Oct 30 22:14:16.165732 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165614    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-home\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-home\") pod \"a9667903-bf26-4c66-b4cf-fecb659a03f2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") "
Oct 30 22:14:16.165732 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165645    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b80c40d-dd47-403d-ae15-2ef2c658a53f-kube-api-access-8rhcd\" (UniqueName: \"kubernetes.io/projected/9b80c40d-dd47-403d-ae15-2ef2c658a53f-kube-api-access-8rhcd\") pod \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") "
Oct 30 22:14:16.165732 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165716    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-dshm\") pod \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") "
Oct 30 22:14:16.165908 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165835    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9667903-bf26-4c66-b4cf-fecb659a03f2-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9667903-bf26-4c66-b4cf-fecb659a03f2-tls-certs\") pod \"a9667903-bf26-4c66-b4cf-fecb659a03f2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") "
Oct 30 22:14:16.165908 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165894    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-model-cache\") pod \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") "
Oct 30 22:14:16.165908 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165837    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-tmp-dir" pod "a9667903-bf26-4c66-b4cf-fecb659a03f2" (UID: "a9667903-bf26-4c66-b4cf-fecb659a03f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.166067 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165890    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-home" pod "a9667903-bf26-4c66-b4cf-fecb659a03f2" (UID: "a9667903-bf26-4c66-b4cf-fecb659a03f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.166067 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165928    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tmp-dir\") pod \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") "
Oct 30 22:14:16.166067 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.165983    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-dshm\") pod \"a9667903-bf26-4c66-b4cf-fecb659a03f2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") "
Oct 30 22:14:16.166067 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166041    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-model-cache\") pod \"a9667903-bf26-4c66-b4cf-fecb659a03f2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") "
Oct 30 22:14:16.166286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166074    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-home\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-home\") pod \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") "
Oct 30 22:14:16.166286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166101    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9667903-bf26-4c66-b4cf-fecb659a03f2-kube-api-access-g7q88\" (UniqueName: \"kubernetes.io/projected/a9667903-bf26-4c66-b4cf-fecb659a03f2-kube-api-access-g7q88\") pod \"a9667903-bf26-4c66-b4cf-fecb659a03f2\" (UID: \"a9667903-bf26-4c66-b4cf-fecb659a03f2\") "
Oct 30 22:14:16.166286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166191    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tls-certs\" (UniqueName: \"kubernetes.io/secret/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tls-certs\") pod \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\" (UID: \"9b80c40d-dd47-403d-ae15-2ef2c658a53f\") "
Oct 30 22:14:16.166286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166194    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tmp-dir" pod "9b80c40d-dd47-403d-ae15-2ef2c658a53f" (UID: "9b80c40d-dd47-403d-ae15-2ef2c658a53f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.166551 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166464    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-home" pod "9b80c40d-dd47-403d-ae15-2ef2c658a53f" (UID: "9b80c40d-dd47-403d-ae15-2ef2c658a53f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.166940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166801    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-model-cache" pod "9b80c40d-dd47-403d-ae15-2ef2c658a53f" (UID: "9b80c40d-dd47-403d-ae15-2ef2c658a53f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.166940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166869    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.166940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166894    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.166940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166909    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.166940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.166921    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.167271 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.167001    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-model-cache" pod "a9667903-bf26-4c66-b4cf-fecb659a03f2" (UID: "a9667903-bf26-4c66-b4cf-fecb659a03f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.168265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.168200    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9667903-bf26-4c66-b4cf-fecb659a03f2-tls-certs" pod "a9667903-bf26-4c66-b4cf-fecb659a03f2" (UID: "a9667903-bf26-4c66-b4cf-fecb659a03f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:16.168452 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.168432    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-dshm" pod "9b80c40d-dd47-403d-ae15-2ef2c658a53f" (UID: "9b80c40d-dd47-403d-ae15-2ef2c658a53f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.168631 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.168607    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b80c40d-dd47-403d-ae15-2ef2c658a53f-kube-api-access-8rhcd" pod "9b80c40d-dd47-403d-ae15-2ef2c658a53f" (UID: "9b80c40d-dd47-403d-ae15-2ef2c658a53f"). InnerVolumeSpecName "kube-api-access-8rhcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:16.169698 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.169666    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tls-certs" pod "9b80c40d-dd47-403d-ae15-2ef2c658a53f" (UID: "9b80c40d-dd47-403d-ae15-2ef2c658a53f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:16.169791 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.169724    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-dshm" pod "a9667903-bf26-4c66-b4cf-fecb659a03f2" (UID: "a9667903-bf26-4c66-b4cf-fecb659a03f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.169791 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.169766    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9667903-bf26-4c66-b4cf-fecb659a03f2-kube-api-access-g7q88" pod "a9667903-bf26-4c66-b4cf-fecb659a03f2" (UID: "a9667903-bf26-4c66-b4cf-fecb659a03f2"). InnerVolumeSpecName "kube-api-access-g7q88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:16.170673 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.170649    5561 generic.go:356] "Generic (PLEG): container finished" podID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerID="e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb" exitCode=0
Oct 30 22:14:16.170752 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.170691    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"
Oct 30 22:14:16.170752 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.170729    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" event={"ID":"9b80c40d-dd47-403d-ae15-2ef2c658a53f","Type":"ContainerDied","Data":"e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb"}
Oct 30 22:14:16.170844 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.170768    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph" event={"ID":"9b80c40d-dd47-403d-ae15-2ef2c658a53f","Type":"ContainerDied","Data":"73df2f855ca703431f57e59b2ac4ddd73d157b454f29baca7cc591c14b32e920"}
Oct 30 22:14:16.170844 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.170788    5561 scope.go:122] "RemoveContainer" containerID="e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb"
Oct 30 22:14:16.172413 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.172388    5561 generic.go:356] "Generic (PLEG): container finished" podID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerID="900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81" exitCode=0
Oct 30 22:14:16.172504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.172433    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"
Oct 30 22:14:16.172504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.172458    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" event={"ID":"a9667903-bf26-4c66-b4cf-fecb659a03f2","Type":"ContainerDied","Data":"900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81"}
Oct 30 22:14:16.172613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.172506    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2" event={"ID":"a9667903-bf26-4c66-b4cf-fecb659a03f2","Type":"ContainerDied","Data":"496fc99e7e765f49fda4b1e4671be874625e8c816aa6049a8f4b8e0981f01f8c"}
Oct 30 22:14:16.188398 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.188377    5561 scope.go:122] "RemoveContainer" containerID="e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb"
Oct 30 22:14:16.188689 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:16.188664    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb\": container with ID starting with e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb not found: ID does not exist" containerID="e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb"
Oct 30 22:14:16.188768 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.188700    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb"} err="failed to get container status \"e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb\": rpc error: code = NotFound desc = could not find container \"e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb\": container with ID starting with e55de2ebfe23efc79da20fb7a6341a80a1aba74df818aa842ca26c11c32593fb not found: ID does not exist"
Oct 30 22:14:16.188768 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.188724    5561 scope.go:122] "RemoveContainer" containerID="900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81"
Oct 30 22:14:16.197116 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.197096    5561 scope.go:122] "RemoveContainer" containerID="900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81"
Oct 30 22:14:16.197447 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:16.197428    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81\": container with ID starting with 900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81 not found: ID does not exist" containerID="900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81"
Oct 30 22:14:16.197517 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.197453    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81"} err="failed to get container status \"900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81\": rpc error: code = NotFound desc = could not find container \"900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81\": container with ID starting with 900a50fb9d65316353e50bb63f3bd9a98714de51eb8a819920f3862fd2c76f81 not found: ID does not exist"
Oct 30 22:14:16.268750 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268712    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rhcd\" (UniqueName: \"kubernetes.io/projected/9b80c40d-dd47-403d-ae15-2ef2c658a53f-kube-api-access-8rhcd\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.268750 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268748    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.268947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268766    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9667903-bf26-4c66-b4cf-fecb659a03f2-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.268947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268779    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b80c40d-dd47-403d-ae15-2ef2c658a53f-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.268947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268795    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.268947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268809    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9667903-bf26-4c66-b4cf-fecb659a03f2-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.268947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268823    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7q88\" (UniqueName: \"kubernetes.io/projected/a9667903-bf26-4c66-b4cf-fecb659a03f2-kube-api-access-g7q88\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.268947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.268840    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b80c40d-dd47-403d-ae15-2ef2c658a53f-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.316261 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.316187    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"]
Oct 30 22:14:16.322650 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.322623    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-w6-557708-kserve-bcc77c68c-f7px2"]
Oct 30 22:14:16.499268 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.499238    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"]
Oct 30 22:14:16.505790 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:16.505763    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-557708/distinct-2-w6-557708-kserve-5ff7d98644-9g4ph"]
Oct 30 22:14:17.846210 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:17.846165    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" path="/var/lib/kubelet/pods/9b80c40d-dd47-403d-ae15-2ef2c658a53f/volumes"
Oct 30 22:14:17.846825 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:17.846802    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" path="/var/lib/kubelet/pods/a9667903-bf26-4c66-b4cf-fecb659a03f2/volumes"
Oct 30 22:14:22.981644 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:22.981601    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:14:27.187393 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:27.187346    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:14:37.564931 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:37.564877    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"]
Oct 30 22:14:37.613446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:37.613415    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"]
Oct 30 22:14:52.610660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.610572    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main" containerID="cri-o://5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878" gracePeriod=45
Oct 30 22:14:52.633143 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.633110    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" containerID="cri-o://cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564" gracePeriod=45
Oct 30 22:14:52.746046 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.746017    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 30 22:14:52.889685 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.889662    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 22:14:52.925712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.925690    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 22:14:52.972730 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.972702    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-kube-api-access-gz9gk\" (UniqueName: \"kubernetes.io/projected/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-kube-api-access-gz9gk\") pod \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") "
Oct 30 22:14:52.972858 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.972765    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tmp-dir\") pod \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") "
Oct 30 22:14:52.972858 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.972805    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-home\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-home\") pod \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") "
Oct 30 22:14:52.972858 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.972848    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-dshm\") pod \"daeb4d95-948f-48a5-9c07-38495186017a\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") "
Oct 30 22:14:52.972995 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.972891    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-model-cache\") pod \"daeb4d95-948f-48a5-9c07-38495186017a\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") "
Oct 30 22:14:52.972995 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.972951    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/daeb4d95-948f-48a5-9c07-38495186017a-tls-certs\" (UniqueName: \"kubernetes.io/secret/daeb4d95-948f-48a5-9c07-38495186017a-tls-certs\") pod \"daeb4d95-948f-48a5-9c07-38495186017a\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") "
Oct 30 22:14:52.972995 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.972984    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tls-certs\" (UniqueName: \"kubernetes.io/secret/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tls-certs\") pod \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") "
Oct 30 22:14:52.973119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973042    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-model-cache\") pod \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") "
Oct 30 22:14:52.973119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973043    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tmp-dir" pod "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" (UID: "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.973119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973066    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-home" pod "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" (UID: "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.973279 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973237    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daeb4d95-948f-48a5-9c07-38495186017a-kube-api-access-dxcwc\" (UniqueName: \"kubernetes.io/projected/daeb4d95-948f-48a5-9c07-38495186017a-kube-api-access-dxcwc\") pod \"daeb4d95-948f-48a5-9c07-38495186017a\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") "
Oct 30 22:14:52.973330 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973300    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-home\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-home\") pod \"daeb4d95-948f-48a5-9c07-38495186017a\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") "
Oct 30 22:14:52.973417 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973386    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-tmp-dir\") pod \"daeb4d95-948f-48a5-9c07-38495186017a\" (UID: \"daeb4d95-948f-48a5-9c07-38495186017a\") "
Oct 30 22:14:52.973474 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973416    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-dshm\") pod \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\" (UID: \"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a\") "
Oct 30 22:14:52.973668 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.973644    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-home" pod "daeb4d95-948f-48a5-9c07-38495186017a" (UID: "daeb4d95-948f-48a5-9c07-38495186017a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.974128 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.974101    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:52.974128 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.974114    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-tmp-dir" pod "daeb4d95-948f-48a5-9c07-38495186017a" (UID: "daeb4d95-948f-48a5-9c07-38495186017a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.974128 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.974128    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:52.974386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.974143    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:52.974386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.974248    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-model-cache" pod "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" (UID: "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.974809 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.974778    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-model-cache" pod "daeb4d95-948f-48a5-9c07-38495186017a" (UID: "daeb4d95-948f-48a5-9c07-38495186017a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.975999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.975810    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-kube-api-access-gz9gk" pod "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" (UID: "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a"). InnerVolumeSpecName "kube-api-access-gz9gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:52.975999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.975916    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daeb4d95-948f-48a5-9c07-38495186017a-tls-certs" pod "daeb4d95-948f-48a5-9c07-38495186017a" (UID: "daeb4d95-948f-48a5-9c07-38495186017a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:52.976487 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.976461    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daeb4d95-948f-48a5-9c07-38495186017a-kube-api-access-dxcwc" pod "daeb4d95-948f-48a5-9c07-38495186017a" (UID: "daeb4d95-948f-48a5-9c07-38495186017a"). InnerVolumeSpecName "kube-api-access-dxcwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:52.976608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.976585    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-dshm" pod "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" (UID: "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.976725 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.976707    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tls-certs" pod "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" (UID: "709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:52.976839 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:52.976821    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-dshm" pod "daeb4d95-948f-48a5-9c07-38495186017a" (UID: "daeb4d95-948f-48a5-9c07-38495186017a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:52.981759 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:52.981729    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:14:53.075793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.075690    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxcwc\" (UniqueName: \"kubernetes.io/projected/daeb4d95-948f-48a5-9c07-38495186017a-kube-api-access-dxcwc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.075793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.075722    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.075793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.075746    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.075793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.075760    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz9gk\" (UniqueName: \"kubernetes.io/projected/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-kube-api-access-gz9gk\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.075793 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.075774    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.088258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.075787    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daeb4d95-948f-48a5-9c07-38495186017a-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.088258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.076144    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daeb4d95-948f-48a5-9c07-38495186017a-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.088258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.076157    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.088258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.076166    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:53.418527 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.418496    5561 generic.go:356] "Generic (PLEG): container finished" podID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerID="5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878" exitCode=0
Oct 30 22:14:53.418724 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.418541    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"
Oct 30 22:14:53.418724 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.418576    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" event={"ID":"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a","Type":"ContainerDied","Data":"5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878"}
Oct 30 22:14:53.418724 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.418622    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k" event={"ID":"709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a","Type":"ContainerDied","Data":"50e3805be5b2a5cb404f0a9b6bc126f392dd2122a1efc8a8909c52163ed66f0e"}
Oct 30 22:14:53.418724 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.418648    5561 scope.go:122] "RemoveContainer" containerID="5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878"
Oct 30 22:14:53.419942 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.419918    5561 generic.go:356] "Generic (PLEG): container finished" podID="daeb4d95-948f-48a5-9c07-38495186017a" containerID="cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564" exitCode=0
Oct 30 22:14:53.420036 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.419963    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"
Oct 30 22:14:53.420036 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.420016    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" event={"ID":"daeb4d95-948f-48a5-9c07-38495186017a","Type":"ContainerDied","Data":"cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564"}
Oct 30 22:14:53.420136 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.420056    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9" event={"ID":"daeb4d95-948f-48a5-9c07-38495186017a","Type":"ContainerDied","Data":"b05078ec8c5db8a2360b50f11ad711c12653cc2e78848dbe58e14a3b600a9fdd"}
Oct 30 22:14:53.429501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.429480    5561 scope.go:122] "RemoveContainer" containerID="5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878"
Oct 30 22:14:53.429808 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:53.429778    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878\": container with ID starting with 5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878 not found: ID does not exist" containerID="5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878"
Oct 30 22:14:53.429919 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.429818    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878"} err="failed to get container status \"5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878\": rpc error: code = NotFound desc = could not find container \"5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878\": container with ID starting with 5126e99a5b734b258c28a7524b6f247290f75d6ccca20132dc5fe6dffd037878 not found: ID does not exist"
Oct 30 22:14:53.429919 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.429843    5561 scope.go:122] "RemoveContainer" containerID="cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564"
Oct 30 22:14:53.437862 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.437844    5561 scope.go:122] "RemoveContainer" containerID="cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564"
Oct 30 22:14:53.438138 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:53.438119    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564\": container with ID starting with cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564 not found: ID does not exist" containerID="cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564"
Oct 30 22:14:53.438190 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.438146    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564"} err="failed to get container status \"cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564\": rpc error: code = NotFound desc = could not find container \"cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564\": container with ID starting with cf22fc60cff1e6639f7c91d8aff87f276bddecc6f57567ed575bb74cce9cc564 not found: ID does not exist"
Oct 30 22:14:53.585327 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.585301    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"]
Oct 30 22:14:53.587578 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.587560    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/premium-w2-c79cb5-kserve-6bd68df849-t5qd9"]
Oct 30 22:14:53.701449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.701389    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"]
Oct 30 22:14:53.715703 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.715679    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c79cb5/distinct-2-w2-c79cb5-kserve-85dd959c8c-n4g8k"]
Oct 30 22:14:53.845250 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.845227    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" path="/var/lib/kubelet/pods/709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a/volumes"
Oct 30 22:14:53.845691 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:53.845675    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daeb4d95-948f-48a5-9c07-38495186017a" path="/var/lib/kubelet/pods/daeb4d95-948f-48a5-9c07-38495186017a/volumes"
Oct 30 22:14:59.118038 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.118005    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"]
Oct 30 22:14:59.118449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.118191    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb" podUID="2aac0e94-fd07-4e94-8c32-959d287f689c" containerName="maas-api" containerID="cri-o://ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04" gracePeriod=30
Oct 30 22:14:59.409237 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.409214    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 22:14:59.441916 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.441885    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2aac0e94-fd07-4e94-8c32-959d287f689c-kube-api-access-2rt29\" (UniqueName: \"kubernetes.io/projected/2aac0e94-fd07-4e94-8c32-959d287f689c-kube-api-access-2rt29\") pod \"2aac0e94-fd07-4e94-8c32-959d287f689c\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") "
Oct 30 22:14:59.442092 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.442075    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls\") pod \"2aac0e94-fd07-4e94-8c32-959d287f689c\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") "
Oct 30 22:14:59.442162 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.442116    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-metrics-certs\") pod \"2aac0e94-fd07-4e94-8c32-959d287f689c\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") "
Oct 30 22:14:59.442222 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.442161    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2aac0e94-fd07-4e94-8c32-959d287f689c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aac0e94-fd07-4e94-8c32-959d287f689c-openshift-service-ca\") pod \"2aac0e94-fd07-4e94-8c32-959d287f689c\" (UID: \"2aac0e94-fd07-4e94-8c32-959d287f689c\") "
Oct 30 22:14:59.442627 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.442589    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2aac0e94-fd07-4e94-8c32-959d287f689c-openshift-service-ca" pod "2aac0e94-fd07-4e94-8c32-959d287f689c" (UID: "2aac0e94-fd07-4e94-8c32-959d287f689c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:59.443997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.443975    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aac0e94-fd07-4e94-8c32-959d287f689c-kube-api-access-2rt29" pod "2aac0e94-fd07-4e94-8c32-959d287f689c" (UID: "2aac0e94-fd07-4e94-8c32-959d287f689c"). InnerVolumeSpecName "kube-api-access-2rt29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:59.443997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.443977    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls" pod "2aac0e94-fd07-4e94-8c32-959d287f689c" (UID: "2aac0e94-fd07-4e94-8c32-959d287f689c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:59.444158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.444138    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-metrics-certs" pod "2aac0e94-fd07-4e94-8c32-959d287f689c" (UID: "2aac0e94-fd07-4e94-8c32-959d287f689c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:59.458802 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.458779    5561 generic.go:356] "Generic (PLEG): container finished" podID="2aac0e94-fd07-4e94-8c32-959d287f689c" containerID="ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04" exitCode=0
Oct 30 22:14:59.458904 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.458820    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"
Oct 30 22:14:59.458904 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.458857    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb" event={"ID":"2aac0e94-fd07-4e94-8c32-959d287f689c","Type":"ContainerDied","Data":"ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04"}
Oct 30 22:14:59.459025 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.458909    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb" event={"ID":"2aac0e94-fd07-4e94-8c32-959d287f689c","Type":"ContainerDied","Data":"88223187ea90e0173d61b6db0f3b18889d8f921c7b5b62280b65ba0b6937dc87"}
Oct 30 22:14:59.459025 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.458932    5561 scope.go:122] "RemoveContainer" containerID="ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04"
Oct 30 22:14:59.468470 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.468448    5561 scope.go:122] "RemoveContainer" containerID="ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04"
Oct 30 22:14:59.468733 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:14:59.468713    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04\": container with ID starting with ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04 not found: ID does not exist" containerID="ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04"
Oct 30 22:14:59.468806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.468738    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04"} err="failed to get container status \"ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04\": rpc error: code = NotFound desc = could not find container \"ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04\": container with ID starting with ee28c1178584c38bb6892fac656b6ace0735794b2ee502a4a6c5c69ebf2b0f04 not found: ID does not exist"
Oct 30 22:14:59.557422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.548293    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:59.557422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.548328    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aac0e94-fd07-4e94-8c32-959d287f689c-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:59.557422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.548345    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aac0e94-fd07-4e94-8c32-959d287f689c-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:59.557422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.548377    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rt29\" (UniqueName: \"kubernetes.io/projected/2aac0e94-fd07-4e94-8c32-959d287f689c-kube-api-access-2rt29\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:59.690945 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.690784    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"]
Oct 30 22:14:59.694100 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.694067    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-557708-6f4d568455-grnqb"]
Oct 30 22:14:59.845404 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:14:59.845367    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2aac0e94-fd07-4e94-8c32-959d287f689c" path="/var/lib/kubelet/pods/2aac0e94-fd07-4e94-8c32-959d287f689c/volumes"
Oct 30 22:15:00.242976 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.242949    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"]
Oct 30 22:15:00.243548 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243531    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243549    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243566    5561 cpu_manager.go:414] "containerMap: removing container" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243571    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243581    5561 cpu_manager.go:414] "containerMap: removing container" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243586    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243595    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243599    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243606    5561 cpu_manager.go:414] "containerMap: removing container" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy"
Oct 30 22:15:00.243611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243611    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243628    5561 cpu_manager.go:414] "containerMap: removing container" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243632    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243667    5561 cpu_manager.go:414] "containerMap: removing container" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243672    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243682    5561 cpu_manager.go:414] "containerMap: removing container" podUID="2aac0e94-fd07-4e94-8c32-959d287f689c" containerName="maas-api"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243686    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2aac0e94-fd07-4e94-8c32-959d287f689c" containerName="maas-api"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243757    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="b887b392-ad21-4cfa-893e-ee93afcfc3d5" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243767    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="daeb4d95-948f-48a5-9c07-38495186017a" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243776    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="2aac0e94-fd07-4e94-8c32-959d287f689c" containerName="maas-api"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243782    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b80c40d-dd47-403d-ae15-2ef2c658a53f" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243790    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="709f5224-60a7-4eb5-b8ea-3e08ed9c2c0a" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243798    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="b22cd2b0-4dcc-4791-9f65-cb9f34f9bb37" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243805    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9667903-bf26-4c66-b4cf-fecb659a03f2" containerName="main"
Oct 30 22:15:00.243884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.243811    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec13631d-151c-4fec-b662-34280fc56a06" containerName="istio-proxy"
Oct 30 22:15:00.248286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.248267    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:00.336753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.336726    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"]
Oct 30 22:15:00.336985 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.336960    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd" podUID="2a77c010-6fce-43c0-a897-77f029793494" containerName="maas-api" containerID="cri-o://1a2213f9abe65552034c1cd7702d4ecfc56df2fbcb9f53cd29bac762f1b9a3ae" gracePeriod=30
Oct 30 22:15:00.357801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.357776    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:15:00.361294 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.361047    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"]
Oct 30 22:15:00.389991 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.389961    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3b6765-170c-410f-84d3-525ef0d40fb8-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-wjxbh\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:00.390211 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.390186    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmh6x\" (UniqueName: \"kubernetes.io/projected/0e3b6765-170c-410f-84d3-525ef0d40fb8-kube-api-access-nmh6x\") pod \"maas-api-key-cleanup-29889975-wjxbh\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:00.467013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.466986    5561 generic.go:356] "Generic (PLEG): container finished" podID="2a77c010-6fce-43c0-a897-77f029793494" containerID="1a2213f9abe65552034c1cd7702d4ecfc56df2fbcb9f53cd29bac762f1b9a3ae" exitCode=0
Oct 30 22:15:00.467145 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.467060    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd" event={"ID":"2a77c010-6fce-43c0-a897-77f029793494","Type":"ContainerDied","Data":"1a2213f9abe65552034c1cd7702d4ecfc56df2fbcb9f53cd29bac762f1b9a3ae"}
Oct 30 22:15:00.491808 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.491776    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3b6765-170c-410f-84d3-525ef0d40fb8-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-wjxbh\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:00.491965 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.491921    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmh6x\" (UniqueName: \"kubernetes.io/projected/0e3b6765-170c-410f-84d3-525ef0d40fb8-kube-api-access-nmh6x\") pod \"maas-api-key-cleanup-29889975-wjxbh\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:00.492609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.492584    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3b6765-170c-410f-84d3-525ef0d40fb8-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-wjxbh\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:00.600243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.600207    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmh6x\" (UniqueName: \"kubernetes.io/projected/0e3b6765-170c-410f-84d3-525ef0d40fb8-kube-api-access-nmh6x\") pod \"maas-api-key-cleanup-29889975-wjxbh\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:00.626688 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.626661    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 22:15:00.698449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.698424    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-metrics-certs\") pod \"2a77c010-6fce-43c0-a897-77f029793494\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") "
Oct 30 22:15:00.698591 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.698517    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a77c010-6fce-43c0-a897-77f029793494-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a77c010-6fce-43c0-a897-77f029793494-openshift-service-ca\") pod \"2a77c010-6fce-43c0-a897-77f029793494\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") "
Oct 30 22:15:00.698591 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.698554    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls\") pod \"2a77c010-6fce-43c0-a897-77f029793494\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") "
Oct 30 22:15:00.698735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.698709    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a77c010-6fce-43c0-a897-77f029793494-kube-api-access-9ccmk\" (UniqueName: \"kubernetes.io/projected/2a77c010-6fce-43c0-a897-77f029793494-kube-api-access-9ccmk\") pod \"2a77c010-6fce-43c0-a897-77f029793494\" (UID: \"2a77c010-6fce-43c0-a897-77f029793494\") "
Oct 30 22:15:00.698848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.698827    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a77c010-6fce-43c0-a897-77f029793494-openshift-service-ca" pod "2a77c010-6fce-43c0-a897-77f029793494" (UID: "2a77c010-6fce-43c0-a897-77f029793494"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:00.699212 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.699191    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a77c010-6fce-43c0-a897-77f029793494-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:00.700627 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.700603    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls" pod "2a77c010-6fce-43c0-a897-77f029793494" (UID: "2a77c010-6fce-43c0-a897-77f029793494"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:15:00.700732 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.700640    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a77c010-6fce-43c0-a897-77f029793494-kube-api-access-9ccmk" pod "2a77c010-6fce-43c0-a897-77f029793494" (UID: "2a77c010-6fce-43c0-a897-77f029793494"). InnerVolumeSpecName "kube-api-access-9ccmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:00.700732 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.700702    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-metrics-certs" pod "2a77c010-6fce-43c0-a897-77f029793494" (UID: "2a77c010-6fce-43c0-a897-77f029793494"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:15:00.826650 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.825878    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:00.826650 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.825924    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ccmk\" (UniqueName: \"kubernetes.io/projected/2a77c010-6fce-43c0-a897-77f029793494-kube-api-access-9ccmk\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:00.826650 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.825940    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a77c010-6fce-43c0-a897-77f029793494-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:00.861381 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:00.861332    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:01.282877 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.282852    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"]
Oct 30 22:15:01.285440 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:15:01.285410    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3b6765_170c_410f_84d3_525ef0d40fb8.slice/crio-68377b7c280825cabd5966c4ec49de81440f81fd6ae6bad253c73f764b72373c WatchSource:0}: Error finding container 68377b7c280825cabd5966c4ec49de81440f81fd6ae6bad253c73f764b72373c: Status 404 returned error can't find the container with id 68377b7c280825cabd5966c4ec49de81440f81fd6ae6bad253c73f764b72373c
Oct 30 22:15:01.287914 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.287400    5561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:15:01.477910 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.477882    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"
Oct 30 22:15:01.478097 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.477898    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd" event={"ID":"2a77c010-6fce-43c0-a897-77f029793494","Type":"ContainerDied","Data":"1bbcc9f8b40a0e24a6567102ede34a097aaa104302f4d3f6261a0555acaf3c6d"}
Oct 30 22:15:01.478097 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.477982    5561 scope.go:122] "RemoveContainer" containerID="1a2213f9abe65552034c1cd7702d4ecfc56df2fbcb9f53cd29bac762f1b9a3ae"
Oct 30 22:15:01.480164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.480141    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" event={"ID":"0e3b6765-170c-410f-84d3-525ef0d40fb8","Type":"ContainerStarted","Data":"68377b7c280825cabd5966c4ec49de81440f81fd6ae6bad253c73f764b72373c"}
Oct 30 22:15:01.692936 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.692901    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"]
Oct 30 22:15:01.732772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.732739    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-c3e2d8-df49956cb-hwmbd"]
Oct 30 22:15:01.846304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:01.846266    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a77c010-6fce-43c0-a897-77f029793494" path="/var/lib/kubelet/pods/2a77c010-6fce-43c0-a897-77f029793494/volumes"
Oct 30 22:15:06.515167 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:06.515088    5561 generic.go:356] "Generic (PLEG): container finished" podID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerID="e78207f6e7df97f4003af61f84f67f8a4ff7ccc1d2731af1d2659e3aa48b66e8" exitCode=60
Oct 30 22:15:06.515167 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:06.515149    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" event={"ID":"0e3b6765-170c-410f-84d3-525ef0d40fb8","Type":"ContainerDied","Data":"e78207f6e7df97f4003af61f84f67f8a4ff7ccc1d2731af1d2659e3aa48b66e8"}
Oct 30 22:15:06.515582 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:06.515396    5561 scope.go:122] "RemoveContainer" containerID="e78207f6e7df97f4003af61f84f67f8a4ff7ccc1d2731af1d2659e3aa48b66e8"
Oct 30 22:15:07.523130 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:07.523097    5561 generic.go:356] "Generic (PLEG): container finished" podID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerID="eb5df819c829cc01f423d9a3d54c7e45e470ac2419475fc6082aa26d6646f443" exitCode=60
Oct 30 22:15:07.523574 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:07.523182    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" event={"ID":"0e3b6765-170c-410f-84d3-525ef0d40fb8","Type":"ContainerDied","Data":"eb5df819c829cc01f423d9a3d54c7e45e470ac2419475fc6082aa26d6646f443"}
Oct 30 22:15:07.523574 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:07.523248    5561 scope.go:122] "RemoveContainer" containerID="e78207f6e7df97f4003af61f84f67f8a4ff7ccc1d2731af1d2659e3aa48b66e8"
Oct 30 22:15:07.523574 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:07.523500    5561 scope.go:122] "RemoveContainer" containerID="eb5df819c829cc01f423d9a3d54c7e45e470ac2419475fc6082aa26d6646f443"
Oct 30 22:15:07.523764 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:07.523735    5561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-wjxbh_odh-ai-gateway-infra(0e3b6765-170c-410f-84d3-525ef0d40fb8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8"
Oct 30 22:15:08.530416 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:08.530385    5561 scope.go:122] "RemoveContainer" containerID="eb5df819c829cc01f423d9a3d54c7e45e470ac2419475fc6082aa26d6646f443"
Oct 30 22:15:08.530808 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:08.530623    5561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-wjxbh_odh-ai-gateway-infra(0e3b6765-170c-410f-84d3-525ef0d40fb8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8"
Oct 30 22:15:10.245575 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:10.245545    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"]
Oct 30 22:15:10.246002 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:10.245768    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy" containerID="cri-o://30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267" gracePeriod=30
Oct 30 22:15:11.171860 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:11.171821    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"]
Oct 30 22:15:11.172112 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:11.172083    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy" containerID="cri-o://7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c" gracePeriod=30
Oct 30 22:15:13.854285 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.854251    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"]
Oct 30 22:15:13.854818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.854803    5561 cpu_manager.go:414] "containerMap: removing container" podUID="2a77c010-6fce-43c0-a897-77f029793494" containerName="maas-api"
Oct 30 22:15:13.854867 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.854819    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a77c010-6fce-43c0-a897-77f029793494" containerName="maas-api"
Oct 30 22:15:13.854926 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.854914    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a77c010-6fce-43c0-a897-77f029793494" containerName="maas-api"
Oct 30 22:15:13.859899 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.859874    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:13.901176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.901145    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"]
Oct 30 22:15:13.903897 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.903869    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smps\" (UniqueName: \"kubernetes.io/projected/a840bba1-a641-441b-966d-772cb050cb0c-kube-api-access-7smps\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:13.904000 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.903913    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-tmp-dir\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:13.904000 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.903942    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-model-cache\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:13.904095 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.904078    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-home\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:13.904131 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.904103    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-dshm\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:13.904167 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.904142    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a840bba1-a641-441b-966d-772cb050cb0c-tls-certs\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:13.949158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.949135    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-954628\"/\"test-model-954628-kserve-self-signed-certs\""
Oct 30 22:15:13.949158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.949152    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-954628\"/\"default-dockercfg-2ftkz\""
Oct 30 22:15:13.964326 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.964306    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-954628\"/\"kube-root-ca.crt\""
Oct 30 22:15:13.964517 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:13.964500    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-954628\"/\"openshift-service-ca.crt\""
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.006488    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-home\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.006544    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-dshm\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.006631    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a840bba1-a641-441b-966d-772cb050cb0c-tls-certs\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.006785    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smps\" (UniqueName: \"kubernetes.io/projected/a840bba1-a641-441b-966d-772cb050cb0c-kube-api-access-7smps\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.006835    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-tmp-dir\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.006872    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-model-cache\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.007426    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-model-cache\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.007782    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-home\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.013086 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.011323    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-tmp-dir\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.015894 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.015866    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-dshm\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.017897 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.017870    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a840bba1-a641-441b-966d-772cb050cb0c-tls-certs\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.076371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.076335    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smps\" (UniqueName: \"kubernetes.io/projected/a840bba1-a641-441b-966d-772cb050cb0c-kube-api-access-7smps\") pod \"test-model-954628-kserve-67cc7c5455-2lwnt\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") " pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.174944 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.174917    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:14.556051 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.556023    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"]
Oct 30 22:15:14.557005 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:15:14.556975    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda840bba1_a641_441b_966d_772cb050cb0c.slice/crio-8a651928186f3e43af1be533f072ec542c3e52aa6e7277d61f01a895fa3c2482 WatchSource:0}: Error finding container 8a651928186f3e43af1be533f072ec542c3e52aa6e7277d61f01a895fa3c2482: Status 404 returned error can't find the container with id 8a651928186f3e43af1be533f072ec542c3e52aa6e7277d61f01a895fa3c2482
Oct 30 22:15:14.576629 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:14.571594    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt" event={"ID":"a840bba1-a641-441b-966d-772cb050cb0c","Type":"ContainerStarted","Data":"8a651928186f3e43af1be533f072ec542c3e52aa6e7277d61f01a895fa3c2482"}
Oct 30 22:15:15.507876 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.507852    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 22:15:15.578675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.578609    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt" event={"ID":"a840bba1-a641-441b-966d-772cb050cb0c","Type":"ContainerStarted","Data":"fdfc52231743aeae1657d1e0cdc748fcb6ef9aca1a1388603ab40e76088747d5"}
Oct 30 22:15:15.579959 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.579929    5561 generic.go:356] "Generic (PLEG): container finished" podID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerID="30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267" exitCode=0
Oct 30 22:15:15.580079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.579967    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"
Oct 30 22:15:15.580079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.579974    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" event={"ID":"de424a99-eb6c-40c9-95d3-d048c456cb53","Type":"ContainerDied","Data":"30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267"}
Oct 30 22:15:15.580079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.580021    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26" event={"ID":"de424a99-eb6c-40c9-95d3-d048c456cb53","Type":"ContainerDied","Data":"c6768433a09f6472bc22596b49c27112f4d1a60af21fbde252bb872cd321821f"}
Oct 30 22:15:15.580079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.580044    5561 scope.go:122] "RemoveContainer" containerID="30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267"
Oct 30 22:15:15.588991 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.588973    5561 scope.go:122] "RemoveContainer" containerID="30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267"
Oct 30 22:15:15.589234 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:15.589216    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267\": container with ID starting with 30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267 not found: ID does not exist" containerID="30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267"
Oct 30 22:15:15.589279 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.589243    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267"} err="failed to get container status \"30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267\": rpc error: code = NotFound desc = could not find container \"30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267\": container with ID starting with 30e6ce9aea9a1f9ae3de0be2d9f0c140b0ca3d163c0bcec59b7a425e96d61267 not found: ID does not exist"
Oct 30 22:15:15.625726 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.625701    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-envoy\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.625851 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.625755    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-kube-api-access-744sg\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-kube-api-access-744sg\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.625851 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.625818    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-podinfo\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.625952 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.625867    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-credential-socket\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.625952 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.625900    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-socket\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.626046 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.625974    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-data\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.626101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.626079    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de424a99-eb6c-40c9-95d3-d048c456cb53-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de424a99-eb6c-40c9-95d3-d048c456cb53-istiod-ca-cert\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.626155 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.626133    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-certs\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.626200 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.626178    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-token\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-token\") pod \"de424a99-eb6c-40c9-95d3-d048c456cb53\" (UID: \"de424a99-eb6c-40c9-95d3-d048c456cb53\") "
Oct 30 22:15:15.626310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.626284    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-credential-socket" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.626609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.626572    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-socket" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.626790 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.626719    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de424a99-eb6c-40c9-95d3-d048c456cb53-istiod-ca-cert" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:15.627059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.627007    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-certs" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.627271 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.627077    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-data" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.627271 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.627139    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.627271 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.627164    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.629070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.629048    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-token" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:15.629902 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.629880    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-podinfo" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:15.630487 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.630466    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-kube-api-access-744sg" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "kube-api-access-744sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:15.630675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.630646    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-envoy" pod "de424a99-eb6c-40c9-95d3-d048c456cb53" (UID: "de424a99-eb6c-40c9-95d3-d048c456cb53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.729405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.728834    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.729405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.728875    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.729405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.728889    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.729405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.728905    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-744sg\" (UniqueName: \"kubernetes.io/projected/de424a99-eb6c-40c9-95d3-d048c456cb53-kube-api-access-744sg\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.729405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.728920    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.729405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.728934    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de424a99-eb6c-40c9-95d3-d048c456cb53-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.729405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.728946    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de424a99-eb6c-40c9-95d3-d048c456cb53-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.797656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.797599    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt" podStartSLOduration=2.797581433 podStartE2EDuration="2.797581433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:15.797057737 +0000 UTC m=+2464.074751213" watchObservedRunningTime="2026-10-30 22:15:15.797581433 +0000 UTC m=+2464.075274905"
Oct 30 22:15:15.995607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:15.995576    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"]
Oct 30 22:15:16.002660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.002633    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26"]
Oct 30 22:15:16.175505 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.175473    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:16.427446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.427394    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 22:15:16.541533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541508    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-kube-api-access-5lnwc\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-kube-api-access-5lnwc\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.541903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541573    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-credential-socket\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.541903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541595    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-socket\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.541903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541675    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-data\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.541903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541693    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-token\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-token\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.541903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541712    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-podinfo\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.541903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541777    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-certs\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.541903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541855    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istiod-ca-cert\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.542231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541900    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-credential-socket" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:16.542231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541932    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-data" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:16.542231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.541937    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-envoy\") pod \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\" (UID: \"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd\") "
Oct 30 22:15:16.542231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.542021    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-socket" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:16.542465 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.542435    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-certs" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:16.542536 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.542481    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.542536 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.542505    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.542536 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.542520    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.542953 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.542928    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istiod-ca-cert" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:16.543991 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.543956    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-kube-api-access-5lnwc" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "kube-api-access-5lnwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:16.544110 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.544090    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-token" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:16.544293 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.544277    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-podinfo" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:16.544508 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.544488    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-envoy" pod "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" (UID: "d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:16.587141 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.587106    5561 generic.go:356] "Generic (PLEG): container finished" podID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerID="7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c" exitCode=0
Oct 30 22:15:16.587269 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.587173    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"
Oct 30 22:15:16.587269 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.587186    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" event={"ID":"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd","Type":"ContainerDied","Data":"7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c"}
Oct 30 22:15:16.587269 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.587243    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv" event={"ID":"d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd","Type":"ContainerDied","Data":"749b9f18af439aae8caa830480ef17a3eb4ee9466674dc8e72ed0988c4421d21"}
Oct 30 22:15:16.587269 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.587266    5561 scope.go:122] "RemoveContainer" containerID="7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c"
Oct 30 22:15:16.597639 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.597277    5561 scope.go:122] "RemoveContainer" containerID="7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c"
Oct 30 22:15:16.597716 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:16.597691    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c\": container with ID starting with 7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c not found: ID does not exist" containerID="7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c"
Oct 30 22:15:16.597772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.597726    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c"} err="failed to get container status \"7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c\": rpc error: code = NotFound desc = could not find container \"7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c\": container with ID starting with 7fbfa6cac9444d558d74acb9b024cf68d453190b0215209c1510f268eebcff8c not found: ID does not exist"
Oct 30 22:15:16.644223 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.644198    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.644325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.644227    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lnwc\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-kube-api-access-5lnwc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.644325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.644242    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.644325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.644256    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.644325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.644270    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.644325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.644284    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:16.754861 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.754800    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"]
Oct 30 22:15:16.758878 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:16.758852    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv"]
Oct 30 22:15:17.203621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.203586    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"]
Oct 30 22:15:17.204139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.204120    5561 cpu_manager.go:414] "containerMap: removing container" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy"
Oct 30 22:15:17.204139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.204137    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy"
Oct 30 22:15:17.204139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.204146    5561 cpu_manager.go:414] "containerMap: removing container" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy"
Oct 30 22:15:17.204324 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.204151    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy"
Oct 30 22:15:17.204324 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.204243    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" containerName="istio-proxy"
Oct 30 22:15:17.204324 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.204252    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" containerName="istio-proxy"
Oct 30 22:15:17.208768 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.208751    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.249964 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.249937    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.250064 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.249977    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.250102 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.250064    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jss5n\" (UniqueName: \"kubernetes.io/projected/83c8ff38-98cb-4724-84e1-44489481ef1d-kube-api-access-jss5n\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.274008 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.273986    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:15:17.277280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.277258    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"]
Oct 30 22:15:17.351443 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.351412    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.351592 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.351558    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.351663 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.351627    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jss5n\" (UniqueName: \"kubernetes.io/projected/83c8ff38-98cb-4724-84e1-44489481ef1d-kube-api-access-jss5n\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.352515 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.352491    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.353893 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.353873    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.426529 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.426497    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jss5n\" (UniqueName: \"kubernetes.io/projected/83c8ff38-98cb-4724-84e1-44489481ef1d-kube-api-access-jss5n\") pod \"maas-api-revoke-sub-8cacb9b5-ns88n\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.522676 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.522609    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:17.845588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.845514    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd" path="/var/lib/kubelet/pods/d1afd3bc-5ed8-4475-8c83-1c5a2cbaabbd/volumes"
Oct 30 22:15:17.846001 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.845986    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de424a99-eb6c-40c9-95d3-d048c456cb53" path="/var/lib/kubelet/pods/de424a99-eb6c-40c9-95d3-d048c456cb53/volumes"
Oct 30 22:15:17.924743 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:17.924712    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"]
Oct 30 22:15:17.930680 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:15:17.930652    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c8ff38_98cb_4724_84e1_44489481ef1d.slice/crio-c971044c3e482dbf06bf12544e007ec9e163dd528692250d60c354f2897324df WatchSource:0}: Error finding container c971044c3e482dbf06bf12544e007ec9e163dd528692250d60c354f2897324df: Status 404 returned error can't find the container with id c971044c3e482dbf06bf12544e007ec9e163dd528692250d60c354f2897324df
Oct 30 22:15:18.623257 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:18.623218    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n" event={"ID":"83c8ff38-98cb-4724-84e1-44489481ef1d","Type":"ContainerStarted","Data":"8c23e2d22f0c8ddecfdb664e1e4f22d982b151b7aca7e63e01137ad64108e6f2"}
Oct 30 22:15:18.623705 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:18.623279    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n" event={"ID":"83c8ff38-98cb-4724-84e1-44489481ef1d","Type":"ContainerStarted","Data":"c971044c3e482dbf06bf12544e007ec9e163dd528692250d60c354f2897324df"}
Oct 30 22:15:18.742710 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:18.742646    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n" podStartSLOduration=1.7426280360000002 podStartE2EDuration="1.742628036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:18.740005398 +0000 UTC m=+2467.017698868" watchObservedRunningTime="2026-10-30 22:15:18.742628036 +0000 UTC m=+2467.020321509"
Oct 30 22:15:19.049721 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:19.049649    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:15:19.630058 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:19.630020    5561 generic.go:356] "Generic (PLEG): container finished" podID="83c8ff38-98cb-4724-84e1-44489481ef1d" containerID="8c23e2d22f0c8ddecfdb664e1e4f22d982b151b7aca7e63e01137ad64108e6f2" exitCode=0
Oct 30 22:15:19.630212 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:19.630107    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n" event={"ID":"83c8ff38-98cb-4724-84e1-44489481ef1d","Type":"ContainerDied","Data":"8c23e2d22f0c8ddecfdb664e1e4f22d982b151b7aca7e63e01137ad64108e6f2"}
Oct 30 22:15:19.839300 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:19.839261    5561 scope.go:122] "RemoveContainer" containerID="eb5df819c829cc01f423d9a3d54c7e45e470ac2419475fc6082aa26d6646f443"
Oct 30 22:15:20.637501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:20.637467    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" event={"ID":"0e3b6765-170c-410f-84d3-525ef0d40fb8","Type":"ContainerStarted","Data":"2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3"}
Oct 30 22:15:20.772261 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:20.772204    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" podStartSLOduration=16.750593786 podStartE2EDuration="20.772183411s" totalImagesPullingTime="4.021589625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:20.768188965 +0000 UTC m=+2469.045882437" watchObservedRunningTime="2026-10-30 22:15:20.772183411 +0000 UTC m=+2469.049876883"
Oct 30 22:15:20.797009 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:20.796816    5561 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3b6765_170c_410f_84d3_525ef0d40fb8.slice/crio-conmon-2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:15:20.797009 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:20.796821    5561 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3b6765_170c_410f_84d3_525ef0d40fb8.slice/crio-2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3b6765_170c_410f_84d3_525ef0d40fb8.slice/crio-conmon-2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:15:20.863607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:20.863572    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:21.003280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.003191    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-openshift-ca\") pod \"83c8ff38-98cb-4724-84e1-44489481ef1d\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") "
Oct 30 22:15:21.003477 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.003301    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83c8ff38-98cb-4724-84e1-44489481ef1d-kube-api-access-jss5n\" (UniqueName: \"kubernetes.io/projected/83c8ff38-98cb-4724-84e1-44489481ef1d-kube-api-access-jss5n\") pod \"83c8ff38-98cb-4724-84e1-44489481ef1d\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") "
Oct 30 22:15:21.003560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.003500    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-xks-ca\") pod \"83c8ff38-98cb-4724-84e1-44489481ef1d\" (UID: \"83c8ff38-98cb-4724-84e1-44489481ef1d\") "
Oct 30 22:15:21.003659 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.003630    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-openshift-ca" pod "83c8ff38-98cb-4724-84e1-44489481ef1d" (UID: "83c8ff38-98cb-4724-84e1-44489481ef1d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:21.004125 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.004096    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-openshift-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:21.005647 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.005619    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-xks-ca" pod "83c8ff38-98cb-4724-84e1-44489481ef1d" (UID: "83c8ff38-98cb-4724-84e1-44489481ef1d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:15:21.006144 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.006117    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c8ff38-98cb-4724-84e1-44489481ef1d-kube-api-access-jss5n" pod "83c8ff38-98cb-4724-84e1-44489481ef1d" (UID: "83c8ff38-98cb-4724-84e1-44489481ef1d"). InnerVolumeSpecName "kube-api-access-jss5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:21.105855 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.105821    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83c8ff38-98cb-4724-84e1-44489481ef1d-maas-api-xks-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:21.105855 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.105853    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jss5n\" (UniqueName: \"kubernetes.io/projected/83c8ff38-98cb-4724-84e1-44489481ef1d-kube-api-access-jss5n\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:21.652307 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.652274    5561 generic.go:356] "Generic (PLEG): container finished" podID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerID="2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3" exitCode=60
Oct 30 22:15:21.652768 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.652372    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" event={"ID":"0e3b6765-170c-410f-84d3-525ef0d40fb8","Type":"ContainerDied","Data":"2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3"}
Oct 30 22:15:21.652768 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.652462    5561 scope.go:122] "RemoveContainer" containerID="eb5df819c829cc01f423d9a3d54c7e45e470ac2419475fc6082aa26d6646f443"
Oct 30 22:15:21.652870 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.652850    5561 scope.go:122] "RemoveContainer" containerID="2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3"
Oct 30 22:15:21.653133 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:21.653098    5561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-wjxbh_odh-ai-gateway-infra(0e3b6765-170c-410f-84d3-525ef0d40fb8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8"
Oct 30 22:15:21.653930 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.653902    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n" event={"ID":"83c8ff38-98cb-4724-84e1-44489481ef1d","Type":"ContainerDied","Data":"c971044c3e482dbf06bf12544e007ec9e163dd528692250d60c354f2897324df"}
Oct 30 22:15:21.654012 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.653948    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c971044c3e482dbf06bf12544e007ec9e163dd528692250d60c354f2897324df"
Oct 30 22:15:21.654012 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.653915    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"
Oct 30 22:15:21.766310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:21.766278    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"]
Oct 30 22:15:22.808175 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:22.808152    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:22.926849 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:22.926769    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e3b6765-170c-410f-84d3-525ef0d40fb8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3b6765-170c-410f-84d3-525ef0d40fb8-maas-api-service-ca\") pod \"0e3b6765-170c-410f-84d3-525ef0d40fb8\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") "
Oct 30 22:15:22.926986 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:22.926878    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e3b6765-170c-410f-84d3-525ef0d40fb8-kube-api-access-nmh6x\" (UniqueName: \"kubernetes.io/projected/0e3b6765-170c-410f-84d3-525ef0d40fb8-kube-api-access-nmh6x\") pod \"0e3b6765-170c-410f-84d3-525ef0d40fb8\" (UID: \"0e3b6765-170c-410f-84d3-525ef0d40fb8\") "
Oct 30 22:15:22.927204 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:22.927182    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3b6765-170c-410f-84d3-525ef0d40fb8-maas-api-service-ca" pod "0e3b6765-170c-410f-84d3-525ef0d40fb8" (UID: "0e3b6765-170c-410f-84d3-525ef0d40fb8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:22.927577 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:22.927556    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3b6765-170c-410f-84d3-525ef0d40fb8-maas-api-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:22.929052 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:22.929017    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3b6765-170c-410f-84d3-525ef0d40fb8-kube-api-access-nmh6x" pod "0e3b6765-170c-410f-84d3-525ef0d40fb8" (UID: "0e3b6765-170c-410f-84d3-525ef0d40fb8"). InnerVolumeSpecName "kube-api-access-nmh6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:22.984009 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:22.983969    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:15:23.028518 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:23.028484    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmh6x\" (UniqueName: \"kubernetes.io/projected/0e3b6765-170c-410f-84d3-525ef0d40fb8-kube-api-access-nmh6x\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:23.670153 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:23.670113    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh" event={"ID":"0e3b6765-170c-410f-84d3-525ef0d40fb8","Type":"ContainerDied","Data":"68377b7c280825cabd5966c4ec49de81440f81fd6ae6bad253c73f764b72373c"}
Oct 30 22:15:23.670312 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:23.670177    5561 scope.go:122] "RemoveContainer" containerID="2a718685aeb53a679dde5dcb44cc7d539ccf5475a20ea88fb931f327d093b2b3"
Oct 30 22:15:23.670312 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:23.670135    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"
Oct 30 22:15:23.836076 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:23.836045    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"]
Oct 30 22:15:23.845452 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:23.845428    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-wjxbh"]
Oct 30 22:15:24.175819 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:24.175784    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:24.189142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:24.189115    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:24.688659 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:24.688626    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:15:25.844828 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:25.844797    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" path="/var/lib/kubelet/pods/0e3b6765-170c-410f-84d3-525ef0d40fb8/volumes"
Oct 30 22:15:31.110560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:31.110531    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"]
Oct 30 22:15:31.110978 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:31.110724    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy" containerID="cri-o://6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9" gracePeriod=30
Oct 30 22:15:31.145599 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:31.145559    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"]
Oct 30 22:15:31.145882 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:31.145853    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy" containerID="cri-o://df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de" gracePeriod=30
Oct 30 22:15:31.151397 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:31.151373    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"]
Oct 30 22:15:31.151556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:31.151538    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy" containerID="cri-o://820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0" gracePeriod=30
Oct 30 22:15:33.356076 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:33.356040    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:15:33.356483 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:33.356102    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:33.464126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:33.464093    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:15:33.464304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:33.464154    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:33.464304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:33.464168    5561 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:15:33.464304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:33.464225    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:36.421578 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.421542    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:15:36.452070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.452044    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:15:36.468681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.468656    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:15:36.499015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.498994    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-data\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.499105 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499050    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9457db31-c6aa-421f-a256-4f74be49b9e0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9457db31-c6aa-421f-a256-4f74be49b9e0-istiod-ca-cert\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.499105 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499091    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-kube-api-access-m6cxt\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-kube-api-access-m6cxt\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.499195 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499132    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-certs\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.499238 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499190    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-credential-socket\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.499238 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499222    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-token\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-token\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.499406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499273    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-data" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.499406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499280    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-kube-api-access-jms6z\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-kube-api-access-jms6z\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.499406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499314    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-socket\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.499406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499364    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istiod-ca-cert\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.499622 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499567    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-credential-socket" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.499622 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499477    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9457db31-c6aa-421f-a256-4f74be49b9e0-istiod-ca-cert" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:36.499622 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499587    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-certs" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.499775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499679    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istiod-ca-cert" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:36.499775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499687    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-envoy\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.499775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499760    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-socket" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.499930 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499777    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-token\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-token\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.499930 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499802    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-podinfo\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.499930 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499893    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-credential-socket\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.499930 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499924    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-socket\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.500139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499954    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-certs\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.500139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.499987    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-envoy\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.500139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500035    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-podinfo\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.500139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500074    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-credential-socket\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.500139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500112    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-token\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-token\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.500439 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500141    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-certs\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.500439 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500167    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istiod-ca-cert\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.500439 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500233    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-kube-api-access-nvph7\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-kube-api-access-nvph7\") pod \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\" (UID: \"77dc14e6-0063-4b6e-95fc-1be5490e2f5e\") "
Oct 30 22:15:36.500439 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500289    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-envoy\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.500439 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500321    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-data\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.500439 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500367    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-podinfo\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.500749 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500448    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-socket\") pod \"9457db31-c6aa-421f-a256-4f74be49b9e0\" (UID: \"9457db31-c6aa-421f-a256-4f74be49b9e0\") "
Oct 30 22:15:36.500749 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500479    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-data\") pod \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\" (UID: \"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d\") "
Oct 30 22:15:36.500749 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500640    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-socket" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500853    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-data" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.500869    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-certs" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501058    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-credential-socket" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501103    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istiod-ca-cert" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501231    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501256    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501273    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501287    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501301    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501315    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501330    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501344    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501379    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9457db31-c6aa-421f-a256-4f74be49b9e0-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501391    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501341    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-socket" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501389    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-certs" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501694    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-credential-socket" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.501739 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.501734    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-data" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.503639 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.503419    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-envoy" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.503639 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.503556    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-token" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.503639 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.503590    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-token" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.504607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.504574    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-kube-api-access-m6cxt" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "kube-api-access-m6cxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.504714 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.504648    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-podinfo" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:36.504714 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.504653    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-podinfo" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:36.504714 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.504698    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-kube-api-access-jms6z" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "kube-api-access-jms6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.504984 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.504721    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-kube-api-access-nvph7" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "kube-api-access-nvph7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.504984 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.504819    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-envoy" pod "9457db31-c6aa-421f-a256-4f74be49b9e0" (UID: "9457db31-c6aa-421f-a256-4f74be49b9e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.504984 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.504856    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-envoy" pod "77dc14e6-0063-4b6e-95fc-1be5490e2f5e" (UID: "77dc14e6-0063-4b6e-95fc-1be5490e2f5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.505133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.505003    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-podinfo" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:36.505133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.505100    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-token" pod "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" (UID: "3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.603152 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603119    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6cxt\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-kube-api-access-m6cxt\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603155    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603172    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jms6z\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-kube-api-access-jms6z\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603183    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603191    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603200    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603208    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603217    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603224    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603232    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603240    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603247    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603256    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvph7\" (UniqueName: \"kubernetes.io/projected/77dc14e6-0063-4b6e-95fc-1be5490e2f5e-kube-api-access-nvph7\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603264    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603272    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603279    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9457db31-c6aa-421f-a256-4f74be49b9e0-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.603744 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.603287    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9457db31-c6aa-421f-a256-4f74be49b9e0-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.749970 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.749886    5561 generic.go:356] "Generic (PLEG): container finished" podID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerID="df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de" exitCode=0
Oct 30 22:15:36.750270 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.750248    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"
Oct 30 22:15:36.750974 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.750943    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" event={"ID":"9457db31-c6aa-421f-a256-4f74be49b9e0","Type":"ContainerDied","Data":"df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de"}
Oct 30 22:15:36.751087 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.751034    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb" event={"ID":"9457db31-c6aa-421f-a256-4f74be49b9e0","Type":"ContainerDied","Data":"d7e3d6f275fcc5df01074d0e3f0427e74a3b792c904ec31b9b245dfda5676844"}
Oct 30 22:15:36.751087 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.751069    5561 scope.go:122] "RemoveContainer" containerID="df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de"
Oct 30 22:15:36.752382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.752327    5561 generic.go:356] "Generic (PLEG): container finished" podID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerID="820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0" exitCode=0
Oct 30 22:15:36.752536 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.752515    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" event={"ID":"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d","Type":"ContainerDied","Data":"820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0"}
Oct 30 22:15:36.752609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.752564    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq" event={"ID":"3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d","Type":"ContainerDied","Data":"e9f3c95b90908184eb685e5995a7b73f7483c0d40bd868f4996d626beff32d46"}
Oct 30 22:15:36.752660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.752641    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"
Oct 30 22:15:36.754382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.754336    5561 generic.go:356] "Generic (PLEG): container finished" podID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerID="6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9" exitCode=0
Oct 30 22:15:36.754493 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.754397    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" event={"ID":"77dc14e6-0063-4b6e-95fc-1be5490e2f5e","Type":"ContainerDied","Data":"6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9"}
Oct 30 22:15:36.754493 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.754438    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb" event={"ID":"77dc14e6-0063-4b6e-95fc-1be5490e2f5e","Type":"ContainerDied","Data":"88c3b0ab77d7059d6c385f7269d35c74ee99a1ac1908202b19a883eb9bb72bf0"}
Oct 30 22:15:36.754614 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.754502    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"
Oct 30 22:15:36.763525 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.763509    5561 scope.go:122] "RemoveContainer" containerID="df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de"
Oct 30 22:15:36.764055 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:36.764035    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de\": container with ID starting with df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de not found: ID does not exist" containerID="df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de"
Oct 30 22:15:36.764198 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.764061    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de"} err="failed to get container status \"df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de\": rpc error: code = NotFound desc = could not find container \"df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de\": container with ID starting with df7ba6ee100a9f91241c51171d06c992dae7ad69ca3be8e71512255adfdac3de not found: ID does not exist"
Oct 30 22:15:36.764198 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.764076    5561 scope.go:122] "RemoveContainer" containerID="820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0"
Oct 30 22:15:36.775302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.775285    5561 scope.go:122] "RemoveContainer" containerID="820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0"
Oct 30 22:15:36.775588 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:36.775570    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0\": container with ID starting with 820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0 not found: ID does not exist" containerID="820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0"
Oct 30 22:15:36.775642 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.775594    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0"} err="failed to get container status \"820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0\": rpc error: code = NotFound desc = could not find container \"820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0\": container with ID starting with 820a1001539d2ccd82ff10289c71f4b28a2e8006443c5dd1cfbe229c5f440ff0 not found: ID does not exist"
Oct 30 22:15:36.775642 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.775620    5561 scope.go:122] "RemoveContainer" containerID="6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9"
Oct 30 22:15:36.786294 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.785412    5561 scope.go:122] "RemoveContainer" containerID="6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9"
Oct 30 22:15:36.786593 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:36.786571    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9\": container with ID starting with 6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9 not found: ID does not exist" containerID="6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9"
Oct 30 22:15:36.786678 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.786597    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9"} err="failed to get container status \"6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9\": rpc error: code = NotFound desc = could not find container \"6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9\": container with ID starting with 6eb55eb5345e005f6a5c5d95efcfc9406d9c6b22fdaf1bdc228d04e3c24f62d9 not found: ID does not exist"
Oct 30 22:15:36.952457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.952422    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"]
Oct 30 22:15:36.956757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:36.956732    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb"]
Oct 30 22:15:37.067606 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:37.067526    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"]
Oct 30 22:15:37.071463 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:37.071432    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb"]
Oct 30 22:15:37.187881 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:37.187852    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"]
Oct 30 22:15:37.193312 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:37.193285    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq"]
Oct 30 22:15:37.845623 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:37.845594    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" path="/var/lib/kubelet/pods/3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d/volumes"
Oct 30 22:15:37.846073 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:37.846057    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" path="/var/lib/kubelet/pods/77dc14e6-0063-4b6e-95fc-1be5490e2f5e/volumes"
Oct 30 22:15:37.846509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:37.846495    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" path="/var/lib/kubelet/pods/9457db31-c6aa-421f-a256-4f74be49b9e0/volumes"
Oct 30 22:15:39.113877 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.113844    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"]
Oct 30 22:15:39.114336 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114323    5561 cpu_manager.go:414] "containerMap: removing container" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114337    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114365    5561 cpu_manager.go:414] "containerMap: removing container" podUID="83c8ff38-98cb-4724-84e1-44489481ef1d" containerName="revoke-keys"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114371    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83c8ff38-98cb-4724-84e1-44489481ef1d" containerName="revoke-keys"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114391    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114396    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114405    5561 cpu_manager.go:414] "containerMap: removing container" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114410    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114417    5561 cpu_manager.go:414] "containerMap: removing container" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114422    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114441    5561 cpu_manager.go:414] "containerMap: removing container" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114445    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114451    5561 cpu_manager.go:414] "containerMap: removing container" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114455    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114521    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="77dc14e6-0063-4b6e-95fc-1be5490e2f5e" containerName="istio-proxy"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114532    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="83c8ff38-98cb-4724-84e1-44489481ef1d" containerName="revoke-keys"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114541    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9457db31-c6aa-421f-a256-4f74be49b9e0" containerName="istio-proxy"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114547    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="3db6bc39-c2df-4ebe-b945-4e2b3ab3d93d" containerName="istio-proxy"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114554    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114563    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.114667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.114569    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e3b6765-170c-410f-84d3-525ef0d40fb8" containerName="cleanup"
Oct 30 22:15:39.121282 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.121262    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.216865 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.216831    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"]
Oct 30 22:15:39.231534 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.231504    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.231689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.231548    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.231689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.231670    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.231786 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.231728    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54430cf3-5169-4e2f-bd82-a50f2cef3861-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.231836 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.231812    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.231961 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.231939    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.232017 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.232001    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.232152 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.232135    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.232218 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.232204    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kvz2\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-kube-api-access-6kvz2\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.333709 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333684    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.333821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333722    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.333821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333761    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.333821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333786    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kvz2\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-kube-api-access-6kvz2\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.333821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333816    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334023 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333833    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334023 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333876    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334023 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333924    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54430cf3-5169-4e2f-bd82-a50f2cef3861-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334023 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.333953    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334228 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.334097    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334228 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.334184    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334228 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.334204    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.334542    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.334654 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.334637    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54430cf3-5169-4e2f-bd82-a50f2cef3861-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.336103 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.336084    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.336458 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.336439    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.391573 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.391550    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.391660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.391571    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kvz2\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-kube-api-access-6kvz2\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.436497 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.436474    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:39.919551 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.919526    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"]
Oct 30 22:15:39.920878 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:15:39.920850    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54430cf3_5169_4e2f_bd82_a50f2cef3861.slice/crio-4d3ecded8db556b1b1fd01a3d52944ae0262f68a87d3a38ad1cfed517529e767 WatchSource:0}: Error finding container 4d3ecded8db556b1b1fd01a3d52944ae0262f68a87d3a38ad1cfed517529e767: Status 404 returned error can't find the container with id 4d3ecded8db556b1b1fd01a3d52944ae0262f68a87d3a38ad1cfed517529e767
Oct 30 22:15:39.923158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.923127    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:39.923221 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.923191    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:39.923255 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:39.923219    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:15:40.782577 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:40.782538    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" event={"ID":"54430cf3-5169-4e2f-bd82-a50f2cef3861","Type":"ContainerStarted","Data":"0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43"}
Oct 30 22:15:40.782977 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:40.782642    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" event={"ID":"54430cf3-5169-4e2f-bd82-a50f2cef3861","Type":"ContainerStarted","Data":"4d3ecded8db556b1b1fd01a3d52944ae0262f68a87d3a38ad1cfed517529e767"}
Oct 30 22:15:40.974155 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:40.974088    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podStartSLOduration=1.974066855 podStartE2EDuration="1.974066855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:40.965243255 +0000 UTC m=+2489.242936729" watchObservedRunningTime="2026-10-30 22:15:40.974066855 +0000 UTC m=+2489.251760330"
Oct 30 22:15:41.437574 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:41.437542    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:41.439116 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:41.439089    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:15:41.439200 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:41.439154    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:15:42.437615 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:42.437579    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:15:42.437976 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:42.437630    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:15:42.733634 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:42.733547    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:15:43.437688 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:43.437651    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:15:43.438097 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:43.437706    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:15:44.437623 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:44.437583    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:15:44.437805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:44.437652    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:15:45.438908 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:45.438861    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:15:45.440142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:45.439664    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:15:46.437187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:46.437153    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:15:46.437383 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:46.437219    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:15:47.441568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:47.441537    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:47.441966 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:47.441819    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:47.442533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:47.442512    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:15:52.981001 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:52.980966    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:15:56.446326 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.446293    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"]
Oct 30 22:15:56.446806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.446600    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j" podUID="a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" containerName="maas-api" containerID="cri-o://2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56" gracePeriod=30
Oct 30 22:15:56.763337 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.763314    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 22:15:56.908913 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.908877    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-openshift-service-ca\") pod \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") "
Oct 30 22:15:56.909119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.909004    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-kube-api-access-hm5gw\" (UniqueName: \"kubernetes.io/projected/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-kube-api-access-hm5gw\") pod \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") "
Oct 30 22:15:56.909119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.909038    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-metrics-certs\") pod \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") "
Oct 30 22:15:56.909253 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.909196    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls\") pod \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\" (UID: \"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935\") "
Oct 30 22:15:56.909395 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.909334    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-openshift-service-ca" pod "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" (UID: "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:56.909718 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.909674    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:56.912219 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.912185    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls" pod "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" (UID: "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:15:56.912314 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.912285    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-kube-api-access-hm5gw" pod "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" (UID: "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935"). InnerVolumeSpecName "kube-api-access-hm5gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:56.912587 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.912564    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-metrics-certs" pod "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" (UID: "a4ca4d2e-b84a-42d0-8636-cfc3c3a55935"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:15:56.936283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.936259    5561 generic.go:356] "Generic (PLEG): container finished" podID="a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" containerID="2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56" exitCode=0
Oct 30 22:15:56.936389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.936301    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"
Oct 30 22:15:56.936389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.936334    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j" event={"ID":"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935","Type":"ContainerDied","Data":"2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56"}
Oct 30 22:15:56.936478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.936398    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j" event={"ID":"a4ca4d2e-b84a-42d0-8636-cfc3c3a55935","Type":"ContainerDied","Data":"4dcda33c68806c315f30d412c3dcc3e1a0f55bad09d922f2fc1c85fc89356b2c"}
Oct 30 22:15:56.936478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.936419    5561 scope.go:122] "RemoveContainer" containerID="2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56"
Oct 30 22:15:56.948520 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.948501    5561 scope.go:122] "RemoveContainer" containerID="2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56"
Oct 30 22:15:56.948774 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:15:56.948751    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56\": container with ID starting with 2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56 not found: ID does not exist" containerID="2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56"
Oct 30 22:15:56.948839 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:56.948787    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56"} err="failed to get container status \"2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56\": rpc error: code = NotFound desc = could not find container \"2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56\": container with ID starting with 2c2ea5b12f607dd1a65b33c4a729933ca93926e0a7238818e51f2080381f7e56 not found: ID does not exist"
Oct 30 22:15:57.015603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:57.012197    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:57.015603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:57.012294    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm5gw\" (UniqueName: \"kubernetes.io/projected/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-kube-api-access-hm5gw\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:57.015603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:57.012312    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:57.116689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:57.116643    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"]
Oct 30 22:15:57.118501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:57.118481    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c79cb5-7f87c477dd-hmt4j"]
Oct 30 22:15:57.844651 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:15:57.844620    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" path="/var/lib/kubelet/pods/a4ca4d2e-b84a-42d0-8636-cfc3c3a55935/volumes"
Oct 30 22:16:05.238999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:05.238967    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"]
Oct 30 22:16:05.239432 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:05.239156    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy" containerID="cri-o://0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43" gracePeriod=30
Oct 30 22:16:05.254478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:05.254452    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"]
Oct 30 22:16:05.254682 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:05.254650    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" containerID="cri-o://fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8" gracePeriod=30
Oct 30 22:16:09.227935 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:09.227890    5561 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:16:09.228443 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:09.227951    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:16:10.518344 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.518281    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:16:10.539607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.539580    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 22:16:10.602823 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.602797    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ad9b344-5973-422c-ae1b-43df366ee690-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad9b344-5973-422c-ae1b-43df366ee690-istio-podinfo\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.602946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.602836    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-istio-token\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-istio-token\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.602946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.602864    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-envoy\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.602946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.602892    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-data\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.602946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.602926    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ad9b344-5973-422c-ae1b-43df366ee690-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad9b344-5973-422c-ae1b-43df366ee690-istiod-ca-cert\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.602946 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.602944    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-data\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.603203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603004    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-socket\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.603203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603043    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-token\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-token\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.603203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603071    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-socket\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.603203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603105    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-certs\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.603203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603135    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-envoy\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.603203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603166    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-data" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.603203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603178    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-kube-api-access-qmjdc\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-kube-api-access-qmjdc\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603289    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-data" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603297    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-podinfo\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603384    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-kube-api-access-6kvz2\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-kube-api-access-6kvz2\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603420    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad9b344-5973-422c-ae1b-43df366ee690-istiod-ca-cert" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603504    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-socket" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603465    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-credential-socket\") pod \"6ad9b344-5973-422c-ae1b-43df366ee690\" (UID: \"6ad9b344-5973-422c-ae1b-43df366ee690\") "
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603593    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-certs\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.603645 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603630    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-credential-socket\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.604019 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603650    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-credential-socket" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.604019 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603759    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54430cf3-5169-4e2f-bd82-a50f2cef3861-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54430cf3-5169-4e2f-bd82-a50f2cef3861-istiod-ca-cert\") pod \"54430cf3-5169-4e2f-bd82-a50f2cef3861\" (UID: \"54430cf3-5169-4e2f-bd82-a50f2cef3861\") "
Oct 30 22:16:10.604019 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.603830    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-socket" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.604556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604275    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.604556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604300    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad9b344-5973-422c-ae1b-43df366ee690-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.604556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604316    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.604556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604329    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.604556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604343    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.604556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604374    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.604556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604420    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-certs" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.604980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604586    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-credential-socket" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.604980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604724    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-certs" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.604980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.604727    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54430cf3-5169-4e2f-bd82-a50f2cef3861-istiod-ca-cert" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:10.605648 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.605615    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-envoy" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.606233 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.606205    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-kube-api-access-qmjdc" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "kube-api-access-qmjdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:10.606233 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.606218    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-podinfo" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:10.606337 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.606208    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ad9b344-5973-422c-ae1b-43df366ee690-istio-podinfo" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:10.606337 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.606242    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-kube-api-access-6kvz2" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "kube-api-access-6kvz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:10.606693 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.606673    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-envoy" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.607102 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.607083    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-token" pod "54430cf3-5169-4e2f-bd82-a50f2cef3861" (UID: "54430cf3-5169-4e2f-bd82-a50f2cef3861"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:10.607164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.607146    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-istio-token" pod "6ad9b344-5973-422c-ae1b-43df366ee690" (UID: "6ad9b344-5973-422c-ae1b-43df366ee690"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:10.705884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705863    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.705884 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705885    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705895    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad9b344-5973-422c-ae1b-43df366ee690-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705905    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmjdc\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-kube-api-access-qmjdc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705919    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705930    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kvz2\" (UniqueName: \"kubernetes.io/projected/54430cf3-5169-4e2f-bd82-a50f2cef3861-kube-api-access-6kvz2\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705938    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705947    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705955    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54430cf3-5169-4e2f-bd82-a50f2cef3861-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705963    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad9b344-5973-422c-ae1b-43df366ee690-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705971    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad9b344-5973-422c-ae1b-43df366ee690-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.706016 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:10.705978    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54430cf3-5169-4e2f-bd82-a50f2cef3861-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:11.017098 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.017061    5561 generic.go:356] "Generic (PLEG): container finished" podID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerID="0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43" exitCode=0
Oct 30 22:16:11.017272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.017110    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"
Oct 30 22:16:11.017272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.017148    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" event={"ID":"54430cf3-5169-4e2f-bd82-a50f2cef3861","Type":"ContainerDied","Data":"0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43"}
Oct 30 22:16:11.017272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.017206    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr" event={"ID":"54430cf3-5169-4e2f-bd82-a50f2cef3861","Type":"ContainerDied","Data":"4d3ecded8db556b1b1fd01a3d52944ae0262f68a87d3a38ad1cfed517529e767"}
Oct 30 22:16:11.017272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.017235    5561 scope.go:122] "RemoveContainer" containerID="0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43"
Oct 30 22:16:11.019188 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.019160    5561 generic.go:356] "Generic (PLEG): container finished" podID="6ad9b344-5973-422c-ae1b-43df366ee690" containerID="fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8" exitCode=0
Oct 30 22:16:11.019310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.019213    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"
Oct 30 22:16:11.019310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.019237    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" event={"ID":"6ad9b344-5973-422c-ae1b-43df366ee690","Type":"ContainerDied","Data":"fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8"}
Oct 30 22:16:11.019310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.019283    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k" event={"ID":"6ad9b344-5973-422c-ae1b-43df366ee690","Type":"ContainerDied","Data":"1c9a98c84ce7d18c2b35efe2eeef7eca34a558f72654dbea7ed33e6367bd9859"}
Oct 30 22:16:11.027919 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.027898    5561 scope.go:122] "RemoveContainer" containerID="0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43"
Oct 30 22:16:11.028188 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:16:11.028167    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43\": container with ID starting with 0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43 not found: ID does not exist" containerID="0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43"
Oct 30 22:16:11.028249 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.028196    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43"} err="failed to get container status \"0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43\": rpc error: code = NotFound desc = could not find container \"0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43\": container with ID starting with 0c5822e51a6266aaeb053e45c785fe387d63204711e58897fa44b63a57235f43 not found: ID does not exist"
Oct 30 22:16:11.028249 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.028213    5561 scope.go:122] "RemoveContainer" containerID="fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8"
Oct 30 22:16:11.036939 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.036918    5561 scope.go:122] "RemoveContainer" containerID="df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5"
Oct 30 22:16:11.044777 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.044759    5561 scope.go:122] "RemoveContainer" containerID="fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8"
Oct 30 22:16:11.045109 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:16:11.045090    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8\": container with ID starting with fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8 not found: ID does not exist" containerID="fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8"
Oct 30 22:16:11.045165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.045123    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8"} err="failed to get container status \"fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8\": rpc error: code = NotFound desc = could not find container \"fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8\": container with ID starting with fd8a6f93296b3693fdbf4420536de4a2dd0f8cfdb279d92665993aaf0573c3c8 not found: ID does not exist"
Oct 30 22:16:11.045165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.045139    5561 scope.go:122] "RemoveContainer" containerID="df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5"
Oct 30 22:16:11.045439 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:16:11.045415    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5\": container with ID starting with df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5 not found: ID does not exist" containerID="df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5"
Oct 30 22:16:11.045513 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.045446    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5"} err="failed to get container status \"df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5\": rpc error: code = NotFound desc = could not find container \"df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5\": container with ID starting with df44641c4022f8640cb7eeec58e39c8d984e41c682f7601635819fb8d9ff3fe5 not found: ID does not exist"
Oct 30 22:16:11.188285 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.188257    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"]
Oct 30 22:16:11.199082 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.199063    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k"]
Oct 30 22:16:11.308049 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.307973    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"]
Oct 30 22:16:11.310062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.310043    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr"]
Oct 30 22:16:11.845754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.845724    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" path="/var/lib/kubelet/pods/54430cf3-5169-4e2f-bd82-a50f2cef3861/volumes"
Oct 30 22:16:11.846171 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:11.846157    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" path="/var/lib/kubelet/pods/6ad9b344-5973-422c-ae1b-43df366ee690/volumes"
Oct 30 22:16:16.926951 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:16.926909    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:16:22.980873 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:16:22.980795    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:16:35.323152 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:16:35.323114    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:16:52.981452 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:16:52.981411    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:17:09.206660 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:09.206622    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:17:20.086694 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.086658    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"]
Oct 30 22:17:20.087162 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.086859    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q" podUID="ac67565e-5766-4d7a-9299-ce582e6e320f" containerName="maas-api" containerID="cri-o://0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04" gracePeriod=30
Oct 30 22:17:20.362455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.362431    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 22:17:20.406321 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.406284    5561 generic.go:356] "Generic (PLEG): container finished" podID="ac67565e-5766-4d7a-9299-ce582e6e320f" containerID="0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04" exitCode=0
Oct 30 22:17:20.406486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.406329    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"
Oct 30 22:17:20.406486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.406327    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q" event={"ID":"ac67565e-5766-4d7a-9299-ce582e6e320f","Type":"ContainerDied","Data":"0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04"}
Oct 30 22:17:20.406486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.406483    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q" event={"ID":"ac67565e-5766-4d7a-9299-ce582e6e320f","Type":"ContainerDied","Data":"521c6f70f767cf3247a86b1c808f2f4fe1d071da7d372d228806ac48deec177f"}
Oct 30 22:17:20.406617 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.406508    5561 scope.go:122] "RemoveContainer" containerID="0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04"
Oct 30 22:17:20.413907 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.413887    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls\") pod \"ac67565e-5766-4d7a-9299-ce582e6e320f\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") "
Oct 30 22:17:20.414015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.413916    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-metrics-certs\") pod \"ac67565e-5766-4d7a-9299-ce582e6e320f\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") "
Oct 30 22:17:20.414059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.414042    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac67565e-5766-4d7a-9299-ce582e6e320f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac67565e-5766-4d7a-9299-ce582e6e320f-openshift-service-ca\") pod \"ac67565e-5766-4d7a-9299-ce582e6e320f\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") "
Oct 30 22:17:20.414134 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.414118    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac67565e-5766-4d7a-9299-ce582e6e320f-kube-api-access-9wpbj\" (UniqueName: \"kubernetes.io/projected/ac67565e-5766-4d7a-9299-ce582e6e320f-kube-api-access-9wpbj\") pod \"ac67565e-5766-4d7a-9299-ce582e6e320f\" (UID: \"ac67565e-5766-4d7a-9299-ce582e6e320f\") "
Oct 30 22:17:20.414478 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.414447    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac67565e-5766-4d7a-9299-ce582e6e320f-openshift-service-ca" pod "ac67565e-5766-4d7a-9299-ce582e6e320f" (UID: "ac67565e-5766-4d7a-9299-ce582e6e320f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:20.414733 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.414702    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac67565e-5766-4d7a-9299-ce582e6e320f-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:20.415605 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.415580    5561 scope.go:122] "RemoveContainer" containerID="0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04"
Oct 30 22:17:20.415924 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:17:20.415894    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04\": container with ID starting with 0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04 not found: ID does not exist" containerID="0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04"
Oct 30 22:17:20.416015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.415934    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04"} err="failed to get container status \"0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04\": rpc error: code = NotFound desc = could not find container \"0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04\": container with ID starting with 0bc9a4713abc86fe16c5ec3c2b49f9ef61bf14eaf79e459bc9fb74994d0bef04 not found: ID does not exist"
Oct 30 22:17:20.416178 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.416154    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls" pod "ac67565e-5766-4d7a-9299-ce582e6e320f" (UID: "ac67565e-5766-4d7a-9299-ce582e6e320f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:20.416330 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.416312    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac67565e-5766-4d7a-9299-ce582e6e320f-kube-api-access-9wpbj" pod "ac67565e-5766-4d7a-9299-ce582e6e320f" (UID: "ac67565e-5766-4d7a-9299-ce582e6e320f"). InnerVolumeSpecName "kube-api-access-9wpbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:20.416413 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.416342    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-metrics-certs" pod "ac67565e-5766-4d7a-9299-ce582e6e320f" (UID: "ac67565e-5766-4d7a-9299-ce582e6e320f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:20.515872 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.515832    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wpbj\" (UniqueName: \"kubernetes.io/projected/ac67565e-5766-4d7a-9299-ce582e6e320f-kube-api-access-9wpbj\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:20.515872 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.515870    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:20.516070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.515885    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac67565e-5766-4d7a-9299-ce582e6e320f-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:20.832168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.832136    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"]
Oct 30 22:17:20.837661 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:20.837636    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-80c366-5b694db4d6-ttp6q"]
Oct 30 22:17:21.845878 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:21.845842    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac67565e-5766-4d7a-9299-ce582e6e320f" path="/var/lib/kubelet/pods/ac67565e-5766-4d7a-9299-ce582e6e320f/volumes"
Oct 30 22:17:22.980261 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:17:22.980225    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:17:29.832738 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:29.832699    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:17:30.578184 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:30.578147    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"]
Oct 30 22:17:30.578400 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:30.578343    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy" containerID="cri-o://fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959" gracePeriod=30
Oct 30 22:17:35.835749 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.835723    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 22:17:35.989254 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989228    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-certs\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989276    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-istio-token\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-istio-token\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989310    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-envoy\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989338    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-data\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989422 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989391    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-credential-socket\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989619 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989446    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-socket\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989619 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989512    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/498354f6-78ca-4228-b51b-f42ad382b63b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/498354f6-78ca-4228-b51b-f42ad382b63b-istio-podinfo\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989619 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989541    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-kube-api-access-x2mvz\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-kube-api-access-x2mvz\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989619 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989540    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-certs" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.989619 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989579    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/498354f6-78ca-4228-b51b-f42ad382b63b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/498354f6-78ca-4228-b51b-f42ad382b63b-istiod-ca-cert\") pod \"498354f6-78ca-4228-b51b-f42ad382b63b\" (UID: \"498354f6-78ca-4228-b51b-f42ad382b63b\") "
Oct 30 22:17:35.989880 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989687    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-credential-socket" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.989880 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989702    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-data" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.989880 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989799    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-socket" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.990044 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.989996    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/498354f6-78ca-4228-b51b-f42ad382b63b-istiod-ca-cert" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:35.990101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.990073    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:35.990101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.990095    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:35.990181 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.990108    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:35.990181 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.990124    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:35.991681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.991658    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-kube-api-access-x2mvz" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "kube-api-access-x2mvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:35.991840 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.991819    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-istio-token" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:35.992023 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.991997    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-envoy" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.992122 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:35.992020    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/498354f6-78ca-4228-b51b-f42ad382b63b-istio-podinfo" pod "498354f6-78ca-4228-b51b-f42ad382b63b" (UID: "498354f6-78ca-4228-b51b-f42ad382b63b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:36.090889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.090856    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/498354f6-78ca-4228-b51b-f42ad382b63b-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.090889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.090885    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2mvz\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-kube-api-access-x2mvz\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.090889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.090896    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/498354f6-78ca-4228-b51b-f42ad382b63b-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.091120 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.090905    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/498354f6-78ca-4228-b51b-f42ad382b63b-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.091120 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.090914    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/498354f6-78ca-4228-b51b-f42ad382b63b-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.506204 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.506169    5561 generic.go:356] "Generic (PLEG): container finished" podID="498354f6-78ca-4228-b51b-f42ad382b63b" containerID="fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959" exitCode=0
Oct 30 22:17:36.506397 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.506220    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"
Oct 30 22:17:36.506397 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.506254    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" event={"ID":"498354f6-78ca-4228-b51b-f42ad382b63b","Type":"ContainerDied","Data":"fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959"}
Oct 30 22:17:36.506397 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.506311    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6" event={"ID":"498354f6-78ca-4228-b51b-f42ad382b63b","Type":"ContainerDied","Data":"c95e86126825508decbc2ea635df3aa9f09beac6d702c74a52e8fdb5fe12b805"}
Oct 30 22:17:36.506397 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.506324    5561 scope.go:122] "RemoveContainer" containerID="fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959"
Oct 30 22:17:36.515942 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.515925    5561 scope.go:122] "RemoveContainer" containerID="010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d"
Oct 30 22:17:36.523893 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.523876    5561 scope.go:122] "RemoveContainer" containerID="fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959"
Oct 30 22:17:36.524131 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:17:36.524109    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959\": container with ID starting with fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959 not found: ID does not exist" containerID="fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959"
Oct 30 22:17:36.524203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.524144    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959"} err="failed to get container status \"fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959\": rpc error: code = NotFound desc = could not find container \"fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959\": container with ID starting with fc9a55be7f8d63faa397298b93fefdcfa1a426412b6dd7d6b931e660dda10959 not found: ID does not exist"
Oct 30 22:17:36.524203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.524170    5561 scope.go:122] "RemoveContainer" containerID="010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d"
Oct 30 22:17:36.524424 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:17:36.524408    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d\": container with ID starting with 010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d not found: ID does not exist" containerID="010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d"
Oct 30 22:17:36.524471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.524430    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d"} err="failed to get container status \"010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d\": rpc error: code = NotFound desc = could not find container \"010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d\": container with ID starting with 010367d943897929fa7384ad0edb20fd46b20c4c44f756bedf7f8f86835bf31d not found: ID does not exist"
Oct 30 22:17:36.673934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.673906    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"]
Oct 30 22:17:36.724286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:36.724260    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6"]
Oct 30 22:17:37.844605 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:37.844571    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" path="/var/lib/kubelet/pods/498354f6-78ca-4228-b51b-f42ad382b63b/volumes"
Oct 30 22:17:52.981241 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:17:52.981149    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:17:55.623772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.623738    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"]
Oct 30 22:17:55.624384 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624342    5561 cpu_manager.go:414] "containerMap: removing container" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy"
Oct 30 22:17:55.624384 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624381    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624396    5561 cpu_manager.go:414] "containerMap: removing container" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624405    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624443    5561 cpu_manager.go:414] "containerMap: removing container" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624450    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624481    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" containerName="maas-api"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624488    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" containerName="maas-api"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624503    5561 cpu_manager.go:414] "containerMap: removing container" podUID="ac67565e-5766-4d7a-9299-ce582e6e320f" containerName="maas-api"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624511    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac67565e-5766-4d7a-9299-ce582e6e320f" containerName="maas-api"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624531    5561 cpu_manager.go:414] "containerMap: removing container" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624539    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624553    5561 cpu_manager.go:414] "containerMap: removing container" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy"
Oct 30 22:17:55.624559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624560    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy"
Oct 30 22:17:55.625142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624686    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac67565e-5766-4d7a-9299-ce582e6e320f" containerName="maas-api"
Oct 30 22:17:55.625142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624705    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy"
Oct 30 22:17:55.625142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624716    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="54430cf3-5169-4e2f-bd82-a50f2cef3861" containerName="istio-proxy"
Oct 30 22:17:55.625142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624726    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy"
Oct 30 22:17:55.625142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624735    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="498354f6-78ca-4228-b51b-f42ad382b63b" containerName="istio-proxy"
Oct 30 22:17:55.625142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624748    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ad9b344-5973-422c-ae1b-43df366ee690" containerName="istio-proxy"
Oct 30 22:17:55.625142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.624761    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4ca4d2e-b84a-42d0-8636-cfc3c3a55935" containerName="maas-api"
Oct 30 22:17:55.628161 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.628141    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.678980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.678946    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e200a26-a686-495a-877a-1676eaaee331-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.679099 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.679046    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e200a26-a686-495a-877a-1676eaaee331-maas-api-xks-ca\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.679099 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.679076    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5pwm\" (UniqueName: \"kubernetes.io/projected/7e200a26-a686-495a-877a-1676eaaee331-kube-api-access-v5pwm\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.685436 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.685412    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"]
Oct 30 22:17:55.732712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.732691    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:17:55.732712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.732697    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:17:55.780009 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.779978    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e200a26-a686-495a-877a-1676eaaee331-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.780133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.780035    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e200a26-a686-495a-877a-1676eaaee331-maas-api-xks-ca\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.780133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.780069    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5pwm\" (UniqueName: \"kubernetes.io/projected/7e200a26-a686-495a-877a-1676eaaee331-kube-api-access-v5pwm\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.780751 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.780730    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e200a26-a686-495a-877a-1676eaaee331-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.782410 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.782388    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e200a26-a686-495a-877a-1676eaaee331-maas-api-xks-ca\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.839519 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.839495    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5pwm\" (UniqueName: \"kubernetes.io/projected/7e200a26-a686-495a-877a-1676eaaee331-kube-api-access-v5pwm\") pod \"maas-api-revoke-sub-15004b6e-hjzlm\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:55.940601 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:55.940541    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:56.312304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:56.312275    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"]
Oct 30 22:17:56.313317 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:17:56.313288    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e200a26_a686_495a_877a_1676eaaee331.slice/crio-f7456be5aa6b9e622a638aee51ca88ebadf1408833311fb2a788c9b654fc9a83 WatchSource:0}: Error finding container f7456be5aa6b9e622a638aee51ca88ebadf1408833311fb2a788c9b654fc9a83: Status 404 returned error can't find the container with id f7456be5aa6b9e622a638aee51ca88ebadf1408833311fb2a788c9b654fc9a83
Oct 30 22:17:56.616257 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:56.616221    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm" event={"ID":"7e200a26-a686-495a-877a-1676eaaee331","Type":"ContainerStarted","Data":"f7456be5aa6b9e622a638aee51ca88ebadf1408833311fb2a788c9b654fc9a83"}
Oct 30 22:17:57.623008 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:57.622973    5561 generic.go:356] "Generic (PLEG): container finished" podID="7e200a26-a686-495a-877a-1676eaaee331" containerID="6d79047d581862520af82fdc37a8afe20dceea93ead49610c43c631ab794e7f4" exitCode=0
Oct 30 22:17:57.623439 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:57.623065    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm" event={"ID":"7e200a26-a686-495a-877a-1676eaaee331","Type":"ContainerDied","Data":"6d79047d581862520af82fdc37a8afe20dceea93ead49610c43c631ab794e7f4"}
Oct 30 22:17:58.766270 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.766243    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:58.813707 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.813680    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e200a26-a686-495a-877a-1676eaaee331-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e200a26-a686-495a-877a-1676eaaee331-maas-api-xks-ca\") pod \"7e200a26-a686-495a-877a-1676eaaee331\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") "
Oct 30 22:17:58.813813 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.813721    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e200a26-a686-495a-877a-1676eaaee331-kube-api-access-v5pwm\" (UniqueName: \"kubernetes.io/projected/7e200a26-a686-495a-877a-1676eaaee331-kube-api-access-v5pwm\") pod \"7e200a26-a686-495a-877a-1676eaaee331\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") "
Oct 30 22:17:58.813813 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.813802    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e200a26-a686-495a-877a-1676eaaee331-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e200a26-a686-495a-877a-1676eaaee331-maas-api-openshift-ca\") pod \"7e200a26-a686-495a-877a-1676eaaee331\" (UID: \"7e200a26-a686-495a-877a-1676eaaee331\") "
Oct 30 22:17:58.814205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.814168    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e200a26-a686-495a-877a-1676eaaee331-maas-api-openshift-ca" pod "7e200a26-a686-495a-877a-1676eaaee331" (UID: "7e200a26-a686-495a-877a-1676eaaee331"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:58.815681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.815654    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e200a26-a686-495a-877a-1676eaaee331-maas-api-xks-ca" pod "7e200a26-a686-495a-877a-1676eaaee331" (UID: "7e200a26-a686-495a-877a-1676eaaee331"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:58.815857 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.815839    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e200a26-a686-495a-877a-1676eaaee331-kube-api-access-v5pwm" pod "7e200a26-a686-495a-877a-1676eaaee331" (UID: "7e200a26-a686-495a-877a-1676eaaee331"). InnerVolumeSpecName "kube-api-access-v5pwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:58.915263 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.915181    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e200a26-a686-495a-877a-1676eaaee331-maas-api-openshift-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.915263 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.915214    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e200a26-a686-495a-877a-1676eaaee331-maas-api-xks-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:58.915263 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:58.915246    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5pwm\" (UniqueName: \"kubernetes.io/projected/7e200a26-a686-495a-877a-1676eaaee331-kube-api-access-v5pwm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:59.636796 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:59.636756    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm" event={"ID":"7e200a26-a686-495a-877a-1676eaaee331","Type":"ContainerDied","Data":"f7456be5aa6b9e622a638aee51ca88ebadf1408833311fb2a788c9b654fc9a83"}
Oct 30 22:17:59.636796 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:59.636774    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"
Oct 30 22:17:59.637001 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:17:59.636805    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7456be5aa6b9e622a638aee51ca88ebadf1408833311fb2a788c9b654fc9a83"
Oct 30 22:18:08.682815 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.682779    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"]
Oct 30 22:18:08.683481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.683326    5561 cpu_manager.go:414] "containerMap: removing container" podUID="7e200a26-a686-495a-877a-1676eaaee331" containerName="revoke-keys"
Oct 30 22:18:08.683481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.683344    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e200a26-a686-495a-877a-1676eaaee331" containerName="revoke-keys"
Oct 30 22:18:08.683481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.683463    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e200a26-a686-495a-877a-1676eaaee331" containerName="revoke-keys"
Oct 30 22:18:08.687027 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.686995    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.749980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.749855    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"]
Oct 30 22:18:08.795785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.795759    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:18:08.795785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.795764    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:18:08.810298 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.810270    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.810475 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.810458    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgd9n\" (UniqueName: \"kubernetes.io/projected/337476fb-9958-4043-abbc-6e36bb5da051-kube-api-access-dgd9n\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.810526 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.810483    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.911915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.911882    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgd9n\" (UniqueName: \"kubernetes.io/projected/337476fb-9958-4043-abbc-6e36bb5da051-kube-api-access-dgd9n\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.911915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.911922    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.912166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.912006    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.912627 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.912600    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.914246 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.914226    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:08.978637 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:08.978575    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgd9n\" (UniqueName: \"kubernetes.io/projected/337476fb-9958-4043-abbc-6e36bb5da051-kube-api-access-dgd9n\") pod \"maas-api-revoke-sub-c97b9e0b-pknqq\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:09.001719 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:09.001694    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:09.374095 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:09.374071    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"]
Oct 30 22:18:09.376256 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:18:09.376231    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337476fb_9958_4043_abbc_6e36bb5da051.slice/crio-09763c4907a9d81522e7607a4bd20b72a1d0f264d7fdd41469460b1decfcaf4b WatchSource:0}: Error finding container 09763c4907a9d81522e7607a4bd20b72a1d0f264d7fdd41469460b1decfcaf4b: Status 404 returned error can't find the container with id 09763c4907a9d81522e7607a4bd20b72a1d0f264d7fdd41469460b1decfcaf4b
Oct 30 22:18:09.696575 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:09.696545    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq" event={"ID":"337476fb-9958-4043-abbc-6e36bb5da051","Type":"ContainerStarted","Data":"ec4d98fdebd6a34372fe6ee9e34e379b7a10dc8dd8d0f64fdf2b4bb70ee083d8"}
Oct 30 22:18:09.696894 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:09.696598    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq" event={"ID":"337476fb-9958-4043-abbc-6e36bb5da051","Type":"ContainerStarted","Data":"09763c4907a9d81522e7607a4bd20b72a1d0f264d7fdd41469460b1decfcaf4b"}
Oct 30 22:18:09.815510 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:09.815453    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq" podStartSLOduration=1.815438863 podStartE2EDuration="1.815438863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:09.814624313 +0000 UTC m=+2638.092317785" watchObservedRunningTime="2026-10-30 22:18:09.815438863 +0000 UTC m=+2638.093132340"
Oct 30 22:18:10.703471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:10.703437    5561 generic.go:356] "Generic (PLEG): container finished" podID="337476fb-9958-4043-abbc-6e36bb5da051" containerID="ec4d98fdebd6a34372fe6ee9e34e379b7a10dc8dd8d0f64fdf2b4bb70ee083d8" exitCode=0
Oct 30 22:18:10.703867 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:10.703522    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq" event={"ID":"337476fb-9958-4043-abbc-6e36bb5da051","Type":"ContainerDied","Data":"ec4d98fdebd6a34372fe6ee9e34e379b7a10dc8dd8d0f64fdf2b4bb70ee083d8"}
Oct 30 22:18:11.841484 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:11.841455    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:11.945113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:11.943242    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-xks-ca\") pod \"337476fb-9958-4043-abbc-6e36bb5da051\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") "
Oct 30 22:18:11.945505 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:11.945481    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/337476fb-9958-4043-abbc-6e36bb5da051-kube-api-access-dgd9n\" (UniqueName: \"kubernetes.io/projected/337476fb-9958-4043-abbc-6e36bb5da051-kube-api-access-dgd9n\") pod \"337476fb-9958-4043-abbc-6e36bb5da051\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") "
Oct 30 22:18:11.945720 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:11.945703    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-openshift-ca\") pod \"337476fb-9958-4043-abbc-6e36bb5da051\" (UID: \"337476fb-9958-4043-abbc-6e36bb5da051\") "
Oct 30 22:18:11.947728 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:11.947701    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-openshift-ca" pod "337476fb-9958-4043-abbc-6e36bb5da051" (UID: "337476fb-9958-4043-abbc-6e36bb5da051"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:11.954666 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:11.954609    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-xks-ca" pod "337476fb-9958-4043-abbc-6e36bb5da051" (UID: "337476fb-9958-4043-abbc-6e36bb5da051"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:11.955942 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:11.955914    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337476fb-9958-4043-abbc-6e36bb5da051-kube-api-access-dgd9n" pod "337476fb-9958-4043-abbc-6e36bb5da051" (UID: "337476fb-9958-4043-abbc-6e36bb5da051"). InnerVolumeSpecName "kube-api-access-dgd9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:12.047465 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:12.047439    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgd9n\" (UniqueName: \"kubernetes.io/projected/337476fb-9958-4043-abbc-6e36bb5da051-kube-api-access-dgd9n\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:12.047465 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:12.047463    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-openshift-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:12.047649 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:12.047474    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/337476fb-9958-4043-abbc-6e36bb5da051-maas-api-xks-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:12.715146 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:12.715115    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"
Oct 30 22:18:12.715314 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:12.715114    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq" event={"ID":"337476fb-9958-4043-abbc-6e36bb5da051","Type":"ContainerDied","Data":"09763c4907a9d81522e7607a4bd20b72a1d0f264d7fdd41469460b1decfcaf4b"}
Oct 30 22:18:12.715314 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:12.715263    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09763c4907a9d81522e7607a4bd20b72a1d0f264d7fdd41469460b1decfcaf4b"
Oct 30 22:18:14.555034 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:14.554999    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:18:22.980155 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:18:22.980117    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:18:30.684327 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:30.684292    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"]
Oct 30 22:18:31.049819 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:31.049735    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:18:41.654596 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:41.654560    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"]
Oct 30 22:18:41.762623 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:41.762592    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"]
Oct 30 22:18:45.614747 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:45.614711    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:18:45.704038 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:45.703996    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt" podUID="a840bba1-a641-441b-966d-772cb050cb0c" containerName="main" containerID="cri-o://fdfc52231743aeae1657d1e0cdc748fcb6ef9aca1a1388603ab40e76088747d5" gracePeriod=45
Oct 30 22:18:45.887826 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:45.887788    5561 generic.go:356] "Generic (PLEG): container finished" podID="a840bba1-a641-441b-966d-772cb050cb0c" containerID="fdfc52231743aeae1657d1e0cdc748fcb6ef9aca1a1388603ab40e76088747d5" exitCode=0
Oct 30 22:18:45.888014 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:45.887868    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt" event={"ID":"a840bba1-a641-441b-966d-772cb050cb0c","Type":"ContainerDied","Data":"fdfc52231743aeae1657d1e0cdc748fcb6ef9aca1a1388603ab40e76088747d5"}
Oct 30 22:18:45.993656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:45.993630    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:18:46.090953 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.090924    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a840bba1-a641-441b-966d-772cb050cb0c-tls-certs\" (UniqueName: \"kubernetes.io/secret/a840bba1-a641-441b-966d-772cb050cb0c-tls-certs\") pod \"a840bba1-a641-441b-966d-772cb050cb0c\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") "
Oct 30 22:18:46.091117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.090961    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-home\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-home\") pod \"a840bba1-a641-441b-966d-772cb050cb0c\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") "
Oct 30 22:18:46.091117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.091000    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-dshm\") pod \"a840bba1-a641-441b-966d-772cb050cb0c\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") "
Oct 30 22:18:46.091117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.091038    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a840bba1-a641-441b-966d-772cb050cb0c-kube-api-access-7smps\" (UniqueName: \"kubernetes.io/projected/a840bba1-a641-441b-966d-772cb050cb0c-kube-api-access-7smps\") pod \"a840bba1-a641-441b-966d-772cb050cb0c\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") "
Oct 30 22:18:46.091117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.091070    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-tmp-dir\") pod \"a840bba1-a641-441b-966d-772cb050cb0c\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") "
Oct 30 22:18:46.091331 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.091139    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-model-cache\") pod \"a840bba1-a641-441b-966d-772cb050cb0c\" (UID: \"a840bba1-a641-441b-966d-772cb050cb0c\") "
Oct 30 22:18:46.091331 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.091301    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-home" pod "a840bba1-a641-441b-966d-772cb050cb0c" (UID: "a840bba1-a641-441b-966d-772cb050cb0c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:46.091527 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.091463    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-tmp-dir" pod "a840bba1-a641-441b-966d-772cb050cb0c" (UID: "a840bba1-a641-441b-966d-772cb050cb0c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:46.091584 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.091538    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-model-cache" pod "a840bba1-a641-441b-966d-772cb050cb0c" (UID: "a840bba1-a641-441b-966d-772cb050cb0c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:46.092052 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.092030    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:46.092121 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.092058    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:46.092121 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.092073    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:46.093198 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.093169    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-dshm" pod "a840bba1-a641-441b-966d-772cb050cb0c" (UID: "a840bba1-a641-441b-966d-772cb050cb0c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:46.093320 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.093276    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a840bba1-a641-441b-966d-772cb050cb0c-tls-certs" pod "a840bba1-a641-441b-966d-772cb050cb0c" (UID: "a840bba1-a641-441b-966d-772cb050cb0c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:46.093393 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.093340    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a840bba1-a641-441b-966d-772cb050cb0c-kube-api-access-7smps" pod "a840bba1-a641-441b-966d-772cb050cb0c" (UID: "a840bba1-a641-441b-966d-772cb050cb0c"). InnerVolumeSpecName "kube-api-access-7smps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:46.193062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.192994    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a840bba1-a641-441b-966d-772cb050cb0c-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:46.193062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.193024    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a840bba1-a641-441b-966d-772cb050cb0c-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:46.193062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.193036    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7smps\" (UniqueName: \"kubernetes.io/projected/a840bba1-a641-441b-966d-772cb050cb0c-kube-api-access-7smps\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:46.655802 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.655763    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7" podUID="e62f24f9-d580-474f-a109-ff1c9e879bcf" containerName="payload-processing" containerID="cri-o://d15efb0fa9b02a1d26ecb887acfc564fb15ce69a70d5821eba20bc417820e9e0" gracePeriod=25
Oct 30 22:18:46.763450 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.763397    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" podUID="55852795-178a-4281-b268-eff4af7382d2" containerName="payload-pre-processing" containerID="cri-o://dadce0648f65c07bef9539281f3be6c0a03ef70f00e97bc41cb470681322a08a" gracePeriod=25
Oct 30 22:18:46.896140 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.896113    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"
Oct 30 22:18:46.896273 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.896112    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt" event={"ID":"a840bba1-a641-441b-966d-772cb050cb0c","Type":"ContainerDied","Data":"8a651928186f3e43af1be533f072ec542c3e52aa6e7277d61f01a895fa3c2482"}
Oct 30 22:18:46.896273 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.896254    5561 scope.go:122] "RemoveContainer" containerID="fdfc52231743aeae1657d1e0cdc748fcb6ef9aca1a1388603ab40e76088747d5"
Oct 30 22:18:46.897602 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.897580    5561 generic.go:356] "Generic (PLEG): container finished" podID="e62f24f9-d580-474f-a109-ff1c9e879bcf" containerID="d15efb0fa9b02a1d26ecb887acfc564fb15ce69a70d5821eba20bc417820e9e0" exitCode=0
Oct 30 22:18:46.897696 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.897653    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7" event={"ID":"e62f24f9-d580-474f-a109-ff1c9e879bcf","Type":"ContainerDied","Data":"d15efb0fa9b02a1d26ecb887acfc564fb15ce69a70d5821eba20bc417820e9e0"}
Oct 30 22:18:46.897757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.897702    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7" event={"ID":"e62f24f9-d580-474f-a109-ff1c9e879bcf","Type":"ContainerDied","Data":"2325411a825e1631651fc17591b2a2e1d1f019af99579e5650f6d524fd726df7"}
Oct 30 22:18:46.897757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.897724    5561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2325411a825e1631651fc17591b2a2e1d1f019af99579e5650f6d524fd726df7"
Oct 30 22:18:46.901766 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:46.901746    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:18:47.001740 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.001672    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e62f24f9-d580-474f-a109-ff1c9e879bcf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62f24f9-d580-474f-a109-ff1c9e879bcf-plugins-config-volume\") pod \"e62f24f9-d580-474f-a109-ff1c9e879bcf\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") "
Oct 30 22:18:47.001864 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.001820    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e62f24f9-d580-474f-a109-ff1c9e879bcf-kube-api-access-shknj\" (UniqueName: \"kubernetes.io/projected/e62f24f9-d580-474f-a109-ff1c9e879bcf-kube-api-access-shknj\") pod \"e62f24f9-d580-474f-a109-ff1c9e879bcf\" (UID: \"e62f24f9-d580-474f-a109-ff1c9e879bcf\") "
Oct 30 22:18:47.002126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.002091    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e62f24f9-d580-474f-a109-ff1c9e879bcf-plugins-config-volume" pod "e62f24f9-d580-474f-a109-ff1c9e879bcf" (UID: "e62f24f9-d580-474f-a109-ff1c9e879bcf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:47.002267 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.002212    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62f24f9-d580-474f-a109-ff1c9e879bcf-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:47.004265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.004238    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e62f24f9-d580-474f-a109-ff1c9e879bcf-kube-api-access-shknj" pod "e62f24f9-d580-474f-a109-ff1c9e879bcf" (UID: "e62f24f9-d580-474f-a109-ff1c9e879bcf"). InnerVolumeSpecName "kube-api-access-shknj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:47.103728 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.103704    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shknj\" (UniqueName: \"kubernetes.io/projected/e62f24f9-d580-474f-a109-ff1c9e879bcf-kube-api-access-shknj\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:47.184948 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.184922    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"]
Oct 30 22:18:47.190941 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.190913    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-954628/test-model-954628-kserve-67cc7c5455-2lwnt"]
Oct 30 22:18:47.845201 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.845172    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a840bba1-a641-441b-966d-772cb050cb0c" path="/var/lib/kubelet/pods/a840bba1-a641-441b-966d-772cb050cb0c/volumes"
Oct 30 22:18:47.902961 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:47.902932    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"
Oct 30 22:18:48.068773 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:48.068740    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"]
Oct 30 22:18:48.072491 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:48.072461    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7"]
Oct 30 22:18:49.845390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:49.845333    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e62f24f9-d580-474f-a109-ff1c9e879bcf" path="/var/lib/kubelet/pods/e62f24f9-d580-474f-a109-ff1c9e879bcf/volumes"
Oct 30 22:18:52.980275 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:18:52.980232    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:18:55.215500 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:18:55.215462    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" podUID="55852795-178a-4281-b268-eff4af7382d2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:19:05.220029 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:05.219982    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" podUID="55852795-178a-4281-b268-eff4af7382d2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:19:07.002420 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.002389    5561 generic.go:356] "Generic (PLEG): container finished" podID="55852795-178a-4281-b268-eff4af7382d2" containerID="dadce0648f65c07bef9539281f3be6c0a03ef70f00e97bc41cb470681322a08a" exitCode=0
Oct 30 22:19:07.002840 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.002459    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" event={"ID":"55852795-178a-4281-b268-eff4af7382d2","Type":"ContainerDied","Data":"dadce0648f65c07bef9539281f3be6c0a03ef70f00e97bc41cb470681322a08a"}
Oct 30 22:19:07.027742 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.027692    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:19:07.086809 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.086788    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55852795-178a-4281-b268-eff4af7382d2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55852795-178a-4281-b268-eff4af7382d2-plugins-config-volume\") pod \"55852795-178a-4281-b268-eff4af7382d2\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") "
Oct 30 22:19:07.086986 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.086973    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55852795-178a-4281-b268-eff4af7382d2-kube-api-access-grkw4\" (UniqueName: \"kubernetes.io/projected/55852795-178a-4281-b268-eff4af7382d2-kube-api-access-grkw4\") pod \"55852795-178a-4281-b268-eff4af7382d2\" (UID: \"55852795-178a-4281-b268-eff4af7382d2\") "
Oct 30 22:19:07.087158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.087134    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55852795-178a-4281-b268-eff4af7382d2-plugins-config-volume" pod "55852795-178a-4281-b268-eff4af7382d2" (UID: "55852795-178a-4281-b268-eff4af7382d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:07.087231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.087214    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55852795-178a-4281-b268-eff4af7382d2-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.088940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.088912    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55852795-178a-4281-b268-eff4af7382d2-kube-api-access-grkw4" pod "55852795-178a-4281-b268-eff4af7382d2" (UID: "55852795-178a-4281-b268-eff4af7382d2"). InnerVolumeSpecName "kube-api-access-grkw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:07.188579 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:07.188551    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grkw4\" (UniqueName: \"kubernetes.io/projected/55852795-178a-4281-b268-eff4af7382d2-kube-api-access-grkw4\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:08.010176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:08.010144    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"
Oct 30 22:19:08.010676 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:08.010142    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p" event={"ID":"55852795-178a-4281-b268-eff4af7382d2","Type":"ContainerDied","Data":"7aaa587147681d36b5b21917d898dba5effe1498159116bca960e3ebb196364b"}
Oct 30 22:19:08.010676 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:08.010296    5561 scope.go:122] "RemoveContainer" containerID="dadce0648f65c07bef9539281f3be6c0a03ef70f00e97bc41cb470681322a08a"
Oct 30 22:19:08.176465 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:08.176434    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"]
Oct 30 22:19:08.227156 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:08.227130    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p"]
Oct 30 22:19:09.845988 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:09.845957    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55852795-178a-4281-b268-eff4af7382d2" path="/var/lib/kubelet/pods/55852795-178a-4281-b268-eff4af7382d2/volumes"
Oct 30 22:19:12.363744 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:12.363716    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:19:12.396726 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:12.396703    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:19:12.882666 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:12.882633    5561 scope.go:122] "RemoveContainer" containerID="d15efb0fa9b02a1d26ecb887acfc564fb15ce69a70d5821eba20bc417820e9e0"
Oct 30 22:19:13.154158 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:13.154078    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:19:22.980102 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:19:22.980022    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:19:31.085681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:31.085648    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"]
Oct 30 22:19:31.191101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:31.191070    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"]
Oct 30 22:19:36.086302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.086258    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c" podUID="6a0c8114-242f-4cdd-9a38-eff32ed82211" containerName="payload-processing" containerID="cri-o://2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692" gracePeriod=25
Oct 30 22:19:36.191731 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.191678    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" podUID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" containerName="payload-pre-processing" containerID="cri-o://3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2" gracePeriod=25
Oct 30 22:19:36.336115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.336089    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:19:36.496997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.496962    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a0c8114-242f-4cdd-9a38-eff32ed82211-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a0c8114-242f-4cdd-9a38-eff32ed82211-plugins-config-volume\") pod \"6a0c8114-242f-4cdd-9a38-eff32ed82211\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") "
Oct 30 22:19:36.497166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.497014    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a0c8114-242f-4cdd-9a38-eff32ed82211-kube-api-access-4nlvp\" (UniqueName: \"kubernetes.io/projected/6a0c8114-242f-4cdd-9a38-eff32ed82211-kube-api-access-4nlvp\") pod \"6a0c8114-242f-4cdd-9a38-eff32ed82211\" (UID: \"6a0c8114-242f-4cdd-9a38-eff32ed82211\") "
Oct 30 22:19:36.497344 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.497319    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a0c8114-242f-4cdd-9a38-eff32ed82211-plugins-config-volume" pod "6a0c8114-242f-4cdd-9a38-eff32ed82211" (UID: "6a0c8114-242f-4cdd-9a38-eff32ed82211"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:36.498955 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.498939    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0c8114-242f-4cdd-9a38-eff32ed82211-kube-api-access-4nlvp" pod "6a0c8114-242f-4cdd-9a38-eff32ed82211" (UID: "6a0c8114-242f-4cdd-9a38-eff32ed82211"). InnerVolumeSpecName "kube-api-access-4nlvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:36.598283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.598256    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a0c8114-242f-4cdd-9a38-eff32ed82211-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:36.598283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:36.598279    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nlvp\" (UniqueName: \"kubernetes.io/projected/6a0c8114-242f-4cdd-9a38-eff32ed82211-kube-api-access-4nlvp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:37.173481 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.173444    5561 generic.go:356] "Generic (PLEG): container finished" podID="6a0c8114-242f-4cdd-9a38-eff32ed82211" containerID="2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692" exitCode=0
Oct 30 22:19:37.173885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.173491    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"
Oct 30 22:19:37.173885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.173524    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c" event={"ID":"6a0c8114-242f-4cdd-9a38-eff32ed82211","Type":"ContainerDied","Data":"2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692"}
Oct 30 22:19:37.173885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.173581    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c" event={"ID":"6a0c8114-242f-4cdd-9a38-eff32ed82211","Type":"ContainerDied","Data":"e555596cfaed5c09064f04f9ac6f4ef2db5fabab463fdabcbe4749fde0552cda"}
Oct 30 22:19:37.173885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.173605    5561 scope.go:122] "RemoveContainer" containerID="2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692"
Oct 30 22:19:37.183509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.183492    5561 scope.go:122] "RemoveContainer" containerID="2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692"
Oct 30 22:19:37.183765 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:19:37.183749    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692\": container with ID starting with 2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692 not found: ID does not exist" containerID="2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692"
Oct 30 22:19:37.183817 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.183773    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692"} err="failed to get container status \"2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692\": rpc error: code = NotFound desc = could not find container \"2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692\": container with ID starting with 2982f4e3fb1e3c6496be54f03f7577a662be1a2a7715898098c523b7c2970692 not found: ID does not exist"
Oct 30 22:19:37.348101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.348069    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"]
Oct 30 22:19:37.398511 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.398483    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c"]
Oct 30 22:19:37.844593 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:37.844559    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a0c8114-242f-4cdd-9a38-eff32ed82211" path="/var/lib/kubelet/pods/6a0c8114-242f-4cdd-9a38-eff32ed82211/volumes"
Oct 30 22:19:39.360004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:39.359965    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" podUID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:19:41.146291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:41.146258    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:19:49.360618 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:49.360578    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" podUID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:19:52.980550 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:19:52.980511    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:19:56.441957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:56.441929    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:19:56.499511 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:56.499448    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdf44047-bb4f-4860-a861-a2c5bfd4358b-kube-api-access-dvrfd\" (UniqueName: \"kubernetes.io/projected/fdf44047-bb4f-4860-a861-a2c5bfd4358b-kube-api-access-dvrfd\") pod \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") "
Oct 30 22:19:56.499625 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:56.499595    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdf44047-bb4f-4860-a861-a2c5bfd4358b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf44047-bb4f-4860-a861-a2c5bfd4358b-plugins-config-volume\") pod \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\" (UID: \"fdf44047-bb4f-4860-a861-a2c5bfd4358b\") "
Oct 30 22:19:56.499986 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:56.499959    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdf44047-bb4f-4860-a861-a2c5bfd4358b-plugins-config-volume" pod "fdf44047-bb4f-4860-a861-a2c5bfd4358b" (UID: "fdf44047-bb4f-4860-a861-a2c5bfd4358b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:56.501387 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:56.501347    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf44047-bb4f-4860-a861-a2c5bfd4358b-kube-api-access-dvrfd" pod "fdf44047-bb4f-4860-a861-a2c5bfd4358b" (UID: "fdf44047-bb4f-4860-a861-a2c5bfd4358b"). InnerVolumeSpecName "kube-api-access-dvrfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:56.600609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:56.600581    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvrfd\" (UniqueName: \"kubernetes.io/projected/fdf44047-bb4f-4860-a861-a2c5bfd4358b-kube-api-access-dvrfd\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:56.600609 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:56.600606    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf44047-bb4f-4860-a861-a2c5bfd4358b-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:57.291102 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.291064    5561 generic.go:356] "Generic (PLEG): container finished" podID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" containerID="3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2" exitCode=0
Oct 30 22:19:57.291272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.291106    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"
Oct 30 22:19:57.291272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.291157    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" event={"ID":"fdf44047-bb4f-4860-a861-a2c5bfd4358b","Type":"ContainerDied","Data":"3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2"}
Oct 30 22:19:57.291272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.291204    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87" event={"ID":"fdf44047-bb4f-4860-a861-a2c5bfd4358b","Type":"ContainerDied","Data":"4f495b55e722bc56c0b8afda6a61b42e8ee9ef3b1afe190a57efb0780a214d25"}
Oct 30 22:19:57.291272 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.291224    5561 scope.go:122] "RemoveContainer" containerID="3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2"
Oct 30 22:19:57.300715 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.300683    5561 scope.go:122] "RemoveContainer" containerID="3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2"
Oct 30 22:19:57.300979 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:19:57.300957    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2\": container with ID starting with 3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2 not found: ID does not exist" containerID="3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2"
Oct 30 22:19:57.301058 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.300987    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2"} err="failed to get container status \"3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2\": rpc error: code = NotFound desc = could not find container \"3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2\": container with ID starting with 3b85493ea6d218ea67209b938d642dbf1797091136c07d8b7d430d6ddf7e03f2 not found: ID does not exist"
Oct 30 22:19:57.456517 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.456485    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"]
Oct 30 22:19:57.506824 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.506800    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87"]
Oct 30 22:19:57.844049 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:19:57.844016    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" path="/var/lib/kubelet/pods/fdf44047-bb4f-4860-a861-a2c5bfd4358b/volumes"
Oct 30 22:20:05.568254 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:05.568217    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"]
Oct 30 22:20:05.674213 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:05.674183    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"]
Oct 30 22:20:10.569492 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.569449    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp" podUID="4db227fc-374f-45bd-aa45-4c5abbe45e74" containerName="payload-processing" containerID="cri-o://cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af" gracePeriod=25
Oct 30 22:20:10.675270 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.675235    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p" podUID="c9780746-8d6a-4e0f-a91d-f14f18d68629" containerName="payload-pre-processing" containerID="cri-o://e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7" gracePeriod=25
Oct 30 22:20:10.837059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.837027    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:20:10.943549 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.943525    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:20:10.976173 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.976144    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4db227fc-374f-45bd-aa45-4c5abbe45e74-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4db227fc-374f-45bd-aa45-4c5abbe45e74-plugins-config-volume\") pod \"4db227fc-374f-45bd-aa45-4c5abbe45e74\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") "
Oct 30 22:20:10.976323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.976290    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4db227fc-374f-45bd-aa45-4c5abbe45e74-kube-api-access-q982r\" (UniqueName: \"kubernetes.io/projected/4db227fc-374f-45bd-aa45-4c5abbe45e74-kube-api-access-q982r\") pod \"4db227fc-374f-45bd-aa45-4c5abbe45e74\" (UID: \"4db227fc-374f-45bd-aa45-4c5abbe45e74\") "
Oct 30 22:20:10.976563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.976532    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4db227fc-374f-45bd-aa45-4c5abbe45e74-plugins-config-volume" pod "4db227fc-374f-45bd-aa45-4c5abbe45e74" (UID: "4db227fc-374f-45bd-aa45-4c5abbe45e74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:10.978222 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:10.978201    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db227fc-374f-45bd-aa45-4c5abbe45e74-kube-api-access-q982r" pod "4db227fc-374f-45bd-aa45-4c5abbe45e74" (UID: "4db227fc-374f-45bd-aa45-4c5abbe45e74"). InnerVolumeSpecName "kube-api-access-q982r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:11.077380 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.077294    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9780746-8d6a-4e0f-a91d-f14f18d68629-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9780746-8d6a-4e0f-a91d-f14f18d68629-plugins-config-volume\") pod \"c9780746-8d6a-4e0f-a91d-f14f18d68629\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") "
Oct 30 22:20:11.077527 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.077508    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9780746-8d6a-4e0f-a91d-f14f18d68629-kube-api-access-6zlzx\" (UniqueName: \"kubernetes.io/projected/c9780746-8d6a-4e0f-a91d-f14f18d68629-kube-api-access-6zlzx\") pod \"c9780746-8d6a-4e0f-a91d-f14f18d68629\" (UID: \"c9780746-8d6a-4e0f-a91d-f14f18d68629\") "
Oct 30 22:20:11.077748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.077716    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9780746-8d6a-4e0f-a91d-f14f18d68629-plugins-config-volume" pod "c9780746-8d6a-4e0f-a91d-f14f18d68629" (UID: "c9780746-8d6a-4e0f-a91d-f14f18d68629"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:11.077886 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.077768    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q982r\" (UniqueName: \"kubernetes.io/projected/4db227fc-374f-45bd-aa45-4c5abbe45e74-kube-api-access-q982r\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:11.077886 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.077786    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4db227fc-374f-45bd-aa45-4c5abbe45e74-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:11.079698 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.079673    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9780746-8d6a-4e0f-a91d-f14f18d68629-kube-api-access-6zlzx" pod "c9780746-8d6a-4e0f-a91d-f14f18d68629" (UID: "c9780746-8d6a-4e0f-a91d-f14f18d68629"). InnerVolumeSpecName "kube-api-access-6zlzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:11.178442 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.178418    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zlzx\" (UniqueName: \"kubernetes.io/projected/c9780746-8d6a-4e0f-a91d-f14f18d68629-kube-api-access-6zlzx\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:11.178442 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.178440    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9780746-8d6a-4e0f-a91d-f14f18d68629-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:11.368804 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.368774    5561 generic.go:356] "Generic (PLEG): container finished" podID="c9780746-8d6a-4e0f-a91d-f14f18d68629" containerID="e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7" exitCode=0
Oct 30 22:20:11.368937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.368821    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"
Oct 30 22:20:11.368937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.368857    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p" event={"ID":"c9780746-8d6a-4e0f-a91d-f14f18d68629","Type":"ContainerDied","Data":"e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7"}
Oct 30 22:20:11.368937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.368914    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p" event={"ID":"c9780746-8d6a-4e0f-a91d-f14f18d68629","Type":"ContainerDied","Data":"5cc2b1f476fa3ebcc68e14a23494a3fd6b0cbf305888b12a95fad2b17c8471b6"}
Oct 30 22:20:11.369095 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.368946    5561 scope.go:122] "RemoveContainer" containerID="e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7"
Oct 30 22:20:11.370162 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.370138    5561 generic.go:356] "Generic (PLEG): container finished" podID="4db227fc-374f-45bd-aa45-4c5abbe45e74" containerID="cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af" exitCode=0
Oct 30 22:20:11.370268 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.370176    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp" event={"ID":"4db227fc-374f-45bd-aa45-4c5abbe45e74","Type":"ContainerDied","Data":"cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af"}
Oct 30 22:20:11.370268 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.370227    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp" event={"ID":"4db227fc-374f-45bd-aa45-4c5abbe45e74","Type":"ContainerDied","Data":"467f1cacb7ac16abc48094641e4a2f43a4d22442060a423ddc4a62a24545f1cd"}
Oct 30 22:20:11.370268 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.370187    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"
Oct 30 22:20:11.384115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.384091    5561 scope.go:122] "RemoveContainer" containerID="e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7"
Oct 30 22:20:11.384407 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:20:11.384385    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7\": container with ID starting with e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7 not found: ID does not exist" containerID="e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7"
Oct 30 22:20:11.384501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.384417    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7"} err="failed to get container status \"e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7\": rpc error: code = NotFound desc = could not find container \"e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7\": container with ID starting with e3098f027f4363893fe178e80eb9a882d1125521094c1c87d5004fccbfcc6fc7 not found: ID does not exist"
Oct 30 22:20:11.384501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.384441    5561 scope.go:122] "RemoveContainer" containerID="cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af"
Oct 30 22:20:11.392156 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.392140    5561 scope.go:122] "RemoveContainer" containerID="cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af"
Oct 30 22:20:11.392418 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:20:11.392403    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af\": container with ID starting with cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af not found: ID does not exist" containerID="cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af"
Oct 30 22:20:11.392471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.392426    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af"} err="failed to get container status \"cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af\": rpc error: code = NotFound desc = could not find container \"cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af\": container with ID starting with cb318a36bc29f41cedceb70e4c80752475eb41d8324d59b0e94ca20a6d1fa7af not found: ID does not exist"
Oct 30 22:20:11.537533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.537502    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"]
Oct 30 22:20:11.540791 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.540764    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp"]
Oct 30 22:20:11.649681 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.649619    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"]
Oct 30 22:20:11.654667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.654645    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p"]
Oct 30 22:20:11.844318 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.844288    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4db227fc-374f-45bd-aa45-4c5abbe45e74" path="/var/lib/kubelet/pods/4db227fc-374f-45bd-aa45-4c5abbe45e74/volumes"
Oct 30 22:20:11.844754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:11.844737    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9780746-8d6a-4e0f-a91d-f14f18d68629" path="/var/lib/kubelet/pods/c9780746-8d6a-4e0f-a91d-f14f18d68629/volumes"
Oct 30 22:20:21.050663 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:21.048190    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"]
Oct 30 22:20:21.096207 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:21.096173    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cacb9b5-ns88n"]
Oct 30 22:20:21.255332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:21.255301    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:20:21.844991 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:21.844948    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83c8ff38-98cb-4724-84e1-44489481ef1d" path="/var/lib/kubelet/pods/83c8ff38-98cb-4724-84e1-44489481ef1d/volumes"
Oct 30 22:20:22.980730 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:20:22.980689    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:20:24.526320 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:24.526289    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"]
Oct 30 22:20:24.526738 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:24.526521    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy" containerID="cri-o://44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9" gracePeriod=30
Oct 30 22:20:26.078024 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:26.077985    5561 patch_prober.go:29] interesting pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:20:26.078412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:26.078048    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:20:29.788548 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.788521    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:20:29.882509 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882484    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-socket\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882517    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-kube-api-access-bznjm\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-kube-api-access-bznjm\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882565    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-credential-socket\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882603    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-envoy\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882670    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fef9544f-6a1c-4289-997c-532d42ba7d38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef9544f-6a1c-4289-997c-532d42ba7d38-istiod-ca-cert\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882690    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-podinfo\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882714    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-token\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-token\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882765    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-socket" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:29.882842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882789    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-certs\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.882842 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882818    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-data\") pod \"fef9544f-6a1c-4289-997c-532d42ba7d38\" (UID: \"fef9544f-6a1c-4289-997c-532d42ba7d38\") "
Oct 30 22:20:29.883160 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.882878    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-credential-socket" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:29.883160 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.883074    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-certs" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:29.883160 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.883102    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fef9544f-6a1c-4289-997c-532d42ba7d38-istiod-ca-cert" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:29.883323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.883246    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.883323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.883269    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.883323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.883284    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef9544f-6a1c-4289-997c-532d42ba7d38-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.883323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.883298    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.883323 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.883251    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-data" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:29.884960 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.884932    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-kube-api-access-bznjm" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "kube-api-access-bznjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:29.884960 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.884947    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-token" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:29.885197 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.885171    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-podinfo" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:29.885238 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.885219    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-envoy" pod "fef9544f-6a1c-4289-997c-532d42ba7d38" (UID: "fef9544f-6a1c-4289-997c-532d42ba7d38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:29.984309 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.984277    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bznjm\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-kube-api-access-bznjm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.984309 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.984307    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.984521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.984322    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.984521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.984335    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:29.984521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:29.984347    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef9544f-6a1c-4289-997c-532d42ba7d38-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:30.478064 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.478027    5561 generic.go:356] "Generic (PLEG): container finished" podID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerID="44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9" exitCode=0
Oct 30 22:20:30.478252 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.478082    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"
Oct 30 22:20:30.478252 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.478110    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" event={"ID":"fef9544f-6a1c-4289-997c-532d42ba7d38","Type":"ContainerDied","Data":"44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9"}
Oct 30 22:20:30.478252 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.478171    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x" event={"ID":"fef9544f-6a1c-4289-997c-532d42ba7d38","Type":"ContainerDied","Data":"691452a493ec881320c8580942eeebda9dee5c02524c785bfab25cf25137806f"}
Oct 30 22:20:30.478252 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.478194    5561 scope.go:122] "RemoveContainer" containerID="44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9"
Oct 30 22:20:30.487948 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.487931    5561 scope.go:122] "RemoveContainer" containerID="cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d"
Oct 30 22:20:30.496045 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.496021    5561 scope.go:122] "RemoveContainer" containerID="44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9"
Oct 30 22:20:30.496291 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:20:30.496273    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9\": container with ID starting with 44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9 not found: ID does not exist" containerID="44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9"
Oct 30 22:20:30.496336 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.496300    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9"} err="failed to get container status \"44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9\": rpc error: code = NotFound desc = could not find container \"44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9\": container with ID starting with 44ecf116f755ef89eaeda6cf7c9a87806ff857aa6bcfa6ba104f95854aaf36b9 not found: ID does not exist"
Oct 30 22:20:30.496336 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.496317    5561 scope.go:122] "RemoveContainer" containerID="cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d"
Oct 30 22:20:30.496593 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:20:30.496575    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d\": container with ID starting with cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d not found: ID does not exist" containerID="cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d"
Oct 30 22:20:30.496646 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.496599    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d"} err="failed to get container status \"cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d\": rpc error: code = NotFound desc = could not find container \"cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d\": container with ID starting with cc6a0ad98fc1f73aef68ae429e06f5482a758cb565df108479f8c7016bf8915d not found: ID does not exist"
Oct 30 22:20:30.643580 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.643540    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"]
Oct 30 22:20:30.694276 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:30.694242    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x"]
Oct 30 22:20:31.844087 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:31.844044    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" path="/var/lib/kubelet/pods/fef9544f-6a1c-4289-997c-532d42ba7d38/volumes"
Oct 30 22:20:46.638094 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:20:46.638058    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:20:52.981542 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:20:52.981458    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:21:05.921196 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:05.921164    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"]
Oct 30 22:21:05.971678 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:05.971648    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"]
Oct 30 22:21:20.941266 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:20.941213    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" containerID="cri-o://703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b" gracePeriod=45
Oct 30 22:21:20.991240 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:20.991207    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main" containerID="cri-o://f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d" gracePeriod=45
Oct 30 22:21:21.236644 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.236621    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 22:21:21.263185 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.263158    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 22:21:21.416608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416574    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-tmp-dir\") pod \"9307b981-13a4-4e6a-836d-e9d0911502ba\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") "
Oct 30 22:21:21.416801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416663    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-tmp-dir\") pod \"4bd4292f-6a46-4aa4-9166-1637614c68f0\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") "
Oct 30 22:21:21.416801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416696    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4bd4292f-6a46-4aa4-9166-1637614c68f0-tls-certs\" (UniqueName: \"kubernetes.io/secret/4bd4292f-6a46-4aa4-9166-1637614c68f0-tls-certs\") pod \"4bd4292f-6a46-4aa4-9166-1637614c68f0\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") "
Oct 30 22:21:21.416801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416721    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-home\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-home\") pod \"4bd4292f-6a46-4aa4-9166-1637614c68f0\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") "
Oct 30 22:21:21.416801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416767    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9307b981-13a4-4e6a-836d-e9d0911502ba-kube-api-access-wc6zc\" (UniqueName: \"kubernetes.io/projected/9307b981-13a4-4e6a-836d-e9d0911502ba-kube-api-access-wc6zc\") pod \"9307b981-13a4-4e6a-836d-e9d0911502ba\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") "
Oct 30 22:21:21.416801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416792    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-model-cache\") pod \"4bd4292f-6a46-4aa4-9166-1637614c68f0\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") "
Oct 30 22:21:21.417075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416850    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-dshm\") pod \"9307b981-13a4-4e6a-836d-e9d0911502ba\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") "
Oct 30 22:21:21.417075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416864    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-tmp-dir" pod "9307b981-13a4-4e6a-836d-e9d0911502ba" (UID: "9307b981-13a4-4e6a-836d-e9d0911502ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.417075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416915    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-home\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-home\") pod \"9307b981-13a4-4e6a-836d-e9d0911502ba\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") "
Oct 30 22:21:21.417075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416932    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-tmp-dir" pod "4bd4292f-6a46-4aa4-9166-1637614c68f0" (UID: "4bd4292f-6a46-4aa4-9166-1637614c68f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.417075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.416970    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9307b981-13a4-4e6a-836d-e9d0911502ba-tls-certs\" (UniqueName: \"kubernetes.io/secret/9307b981-13a4-4e6a-836d-e9d0911502ba-tls-certs\") pod \"9307b981-13a4-4e6a-836d-e9d0911502ba\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") "
Oct 30 22:21:21.417075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417015    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-model-cache\") pod \"9307b981-13a4-4e6a-836d-e9d0911502ba\" (UID: \"9307b981-13a4-4e6a-836d-e9d0911502ba\") "
Oct 30 22:21:21.417075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417068    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-dshm\") pod \"4bd4292f-6a46-4aa4-9166-1637614c68f0\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") "
Oct 30 22:21:21.417440 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417081    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-home" pod "4bd4292f-6a46-4aa4-9166-1637614c68f0" (UID: "4bd4292f-6a46-4aa4-9166-1637614c68f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.417440 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417133    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bd4292f-6a46-4aa4-9166-1637614c68f0-kube-api-access-gj4bp\" (UniqueName: \"kubernetes.io/projected/4bd4292f-6a46-4aa4-9166-1637614c68f0-kube-api-access-gj4bp\") pod \"4bd4292f-6a46-4aa4-9166-1637614c68f0\" (UID: \"4bd4292f-6a46-4aa4-9166-1637614c68f0\") "
Oct 30 22:21:21.417440 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417167    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-model-cache" pod "4bd4292f-6a46-4aa4-9166-1637614c68f0" (UID: "4bd4292f-6a46-4aa4-9166-1637614c68f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.417440 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417409    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-model-cache" pod "9307b981-13a4-4e6a-836d-e9d0911502ba" (UID: "9307b981-13a4-4e6a-836d-e9d0911502ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.417670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417526    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.417670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417547    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.417670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417561    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.417670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417575    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.417670 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417587    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.418126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.417986    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-home" pod "9307b981-13a4-4e6a-836d-e9d0911502ba" (UID: "9307b981-13a4-4e6a-836d-e9d0911502ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.419892 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.419863    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-dshm" pod "9307b981-13a4-4e6a-836d-e9d0911502ba" (UID: "9307b981-13a4-4e6a-836d-e9d0911502ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.420163 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.420115    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bd4292f-6a46-4aa4-9166-1637614c68f0-kube-api-access-gj4bp" pod "4bd4292f-6a46-4aa4-9166-1637614c68f0" (UID: "4bd4292f-6a46-4aa4-9166-1637614c68f0"). InnerVolumeSpecName "kube-api-access-gj4bp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:21.420284 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.420262    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-dshm" pod "4bd4292f-6a46-4aa4-9166-1637614c68f0" (UID: "4bd4292f-6a46-4aa4-9166-1637614c68f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:21.420396 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.420316    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bd4292f-6a46-4aa4-9166-1637614c68f0-tls-certs" pod "4bd4292f-6a46-4aa4-9166-1637614c68f0" (UID: "4bd4292f-6a46-4aa4-9166-1637614c68f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:21.420774 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.420752    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9307b981-13a4-4e6a-836d-e9d0911502ba-tls-certs" pod "9307b981-13a4-4e6a-836d-e9d0911502ba" (UID: "9307b981-13a4-4e6a-836d-e9d0911502ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:21.421203 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.421185    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9307b981-13a4-4e6a-836d-e9d0911502ba-kube-api-access-wc6zc" pod "9307b981-13a4-4e6a-836d-e9d0911502ba" (UID: "9307b981-13a4-4e6a-836d-e9d0911502ba"). InnerVolumeSpecName "kube-api-access-wc6zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:21.519117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.519032    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.519117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.519062    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9307b981-13a4-4e6a-836d-e9d0911502ba-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.519117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.519076    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9307b981-13a4-4e6a-836d-e9d0911502ba-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.519117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.519087    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bd4292f-6a46-4aa4-9166-1637614c68f0-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.519117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.519100    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj4bp\" (UniqueName: \"kubernetes.io/projected/4bd4292f-6a46-4aa4-9166-1637614c68f0-kube-api-access-gj4bp\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.519117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.519114    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bd4292f-6a46-4aa4-9166-1637614c68f0-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.519433 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.519127    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc6zc\" (UniqueName: \"kubernetes.io/projected/9307b981-13a4-4e6a-836d-e9d0911502ba-kube-api-access-wc6zc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:21.761719 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.761684    5561 generic.go:356] "Generic (PLEG): container finished" podID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerID="f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d" exitCode=0
Oct 30 22:21:21.761896 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.761723    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"
Oct 30 22:21:21.761896 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.761770    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" event={"ID":"9307b981-13a4-4e6a-836d-e9d0911502ba","Type":"ContainerDied","Data":"f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d"}
Oct 30 22:21:21.761896 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.761819    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc" event={"ID":"9307b981-13a4-4e6a-836d-e9d0911502ba","Type":"ContainerDied","Data":"26650fc7ddac0760ff69f189c4654601bee21c2080c57f71d0eda3eb4377bf15"}
Oct 30 22:21:21.761896 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.761844    5561 scope.go:122] "RemoveContainer" containerID="f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d"
Oct 30 22:21:21.763126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.763101    5561 generic.go:356] "Generic (PLEG): container finished" podID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerID="703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b" exitCode=0
Oct 30 22:21:21.763221 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.763153    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"
Oct 30 22:21:21.763221 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.763168    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" event={"ID":"4bd4292f-6a46-4aa4-9166-1637614c68f0","Type":"ContainerDied","Data":"703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b"}
Oct 30 22:21:21.763221 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.763193    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" event={"ID":"4bd4292f-6a46-4aa4-9166-1637614c68f0","Type":"ContainerDied","Data":"68cb6feb063101649799cd99daa54250deb2ae32f7d848708cf50604a12ebf07"}
Oct 30 22:21:21.775055 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.775033    5561 scope.go:122] "RemoveContainer" containerID="f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d"
Oct 30 22:21:21.775460 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:21:21.775421    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d\": container with ID starting with f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d not found: ID does not exist" containerID="f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d"
Oct 30 22:21:21.775602 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.775578    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d"} err="failed to get container status \"f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d\": rpc error: code = NotFound desc = could not find container \"f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d\": container with ID starting with f87555bdebe34ffc933c17bd76170a719eee5b3110585ef5145e948136831d5d not found: ID does not exist"
Oct 30 22:21:21.775602 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.775601    5561 scope.go:122] "RemoveContainer" containerID="703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b"
Oct 30 22:21:21.783278 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.783258    5561 scope.go:122] "RemoveContainer" containerID="703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b"
Oct 30 22:21:21.783531 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:21:21.783510    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b\": container with ID starting with 703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b not found: ID does not exist" containerID="703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b"
Oct 30 22:21:21.783600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.783542    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b"} err="failed to get container status \"703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b\": rpc error: code = NotFound desc = could not find container \"703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b\": container with ID starting with 703f9aa83ad772ceaf20fdc9eb6dc3caf4372596b301df3acdde2a9dc3d6d52b not found: ID does not exist"
Oct 30 22:21:21.927168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.927141    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"]
Oct 30 22:21:21.977530 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:21.977506    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-w3-bc009f-kserve-6f6dd75bc4-l2njc"]
Oct 30 22:21:22.087302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:22.087241    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"]
Oct 30 22:21:22.094674 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:22.094638    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z"]
Oct 30 22:21:22.214532 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:22.214497    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc009f/distinct-2-w3-bc009f-kserve-67c5f8cdf7-89k7z" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:22.980296 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:21:22.980260    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:21:23.844656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:23.844612    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" path="/var/lib/kubelet/pods/4bd4292f-6a46-4aa4-9166-1637614c68f0/volumes"
Oct 30 22:21:23.845109 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:23.845091    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" path="/var/lib/kubelet/pods/9307b981-13a4-4e6a-836d-e9d0911502ba/volumes"
Oct 30 22:21:31.776175 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:31.776132    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"]
Oct 30 22:21:31.776704 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:31.776331    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c" podUID="4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" containerName="maas-api" containerID="cri-o://8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45" gracePeriod=30
Oct 30 22:21:32.065940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.065914    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 22:21:32.117043 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.117016    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-kube-api-access-x6gc5\" (UniqueName: \"kubernetes.io/projected/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-kube-api-access-x6gc5\") pod \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") "
Oct 30 22:21:32.117169 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.117072    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls\") pod \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") "
Oct 30 22:21:32.117169 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.117113    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-openshift-service-ca\") pod \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") "
Oct 30 22:21:32.117169 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.117165    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-metrics-certs\") pod \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\" (UID: \"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2\") "
Oct 30 22:21:32.117519 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.117491    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-openshift-service-ca" pod "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" (UID: "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:32.119147 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.119121    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-kube-api-access-x6gc5" pod "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" (UID: "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2"). InnerVolumeSpecName "kube-api-access-x6gc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:32.119147 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.119137    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls" pod "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" (UID: "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:32.119447 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.119423    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-metrics-certs" pod "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" (UID: "4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:32.218806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.218775    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6gc5\" (UniqueName: \"kubernetes.io/projected/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-kube-api-access-x6gc5\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:32.218806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.218804    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:32.218806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.218813    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:32.219042 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.218824    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:32.362892 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.362810    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:21:32.822262 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.822224    5561 generic.go:356] "Generic (PLEG): container finished" podID="4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" containerID="8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45" exitCode=0
Oct 30 22:21:32.822718 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.822269    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"
Oct 30 22:21:32.822718 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.822315    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c" event={"ID":"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2","Type":"ContainerDied","Data":"8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45"}
Oct 30 22:21:32.822718 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.822389    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c" event={"ID":"4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2","Type":"ContainerDied","Data":"7e9b0eec2348d1eea93ce20e381a20ee57daa0a678b5bf9333ee878a636ae286"}
Oct 30 22:21:32.822718 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.822414    5561 scope.go:122] "RemoveContainer" containerID="8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45"
Oct 30 22:21:32.832261 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.832242    5561 scope.go:122] "RemoveContainer" containerID="8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45"
Oct 30 22:21:32.832552 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:21:32.832529    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45\": container with ID starting with 8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45 not found: ID does not exist" containerID="8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45"
Oct 30 22:21:32.832603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.832565    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45"} err="failed to get container status \"8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45\": rpc error: code = NotFound desc = could not find container \"8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45\": container with ID starting with 8506e59dcab8759824805e805e3e5938e6900b8b2539a265e44aa0fd52757c45 not found: ID does not exist"
Oct 30 22:21:32.988600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.988569    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"]
Oct 30 22:21:32.992494 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:32.992465    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-bc009f-569784ffd-25z7c"]
Oct 30 22:21:33.844033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:33.843998    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" path="/var/lib/kubelet/pods/4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2/volumes"
Oct 30 22:21:40.689090 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:40.689039    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"]
Oct 30 22:21:40.689591 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:40.689321    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy" containerID="cri-o://e9c432a74870040f54a270481dd4e041a93941f747034aa8c3afe149e512f357" gracePeriod=30
Oct 30 22:21:45.887549 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:45.887512    5561 generic.go:356] "Generic (PLEG): container finished" podID="f40424dd-b515-4029-b062-02fbae312213" containerID="e9c432a74870040f54a270481dd4e041a93941f747034aa8c3afe149e512f357" exitCode=0
Oct 30 22:21:45.887933 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:45.887573    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" event={"ID":"f40424dd-b515-4029-b062-02fbae312213","Type":"ContainerDied","Data":"e9c432a74870040f54a270481dd4e041a93941f747034aa8c3afe149e512f357"}
Oct 30 22:21:45.938434 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:45.938411    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 22:21:46.043437 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.043371    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7567487b8b-l7ph6"]
Oct 30 22:21:46.043571 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.043545    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-7567487b8b-l7ph6" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerName="manager" containerID="cri-o://37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e" gracePeriod=30
Oct 30 22:21:46.058634 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058605    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f40424dd-b515-4029-b062-02fbae312213-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f40424dd-b515-4029-b062-02fbae312213-istiod-ca-cert\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.058761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058645    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f40424dd-b515-4029-b062-02fbae312213-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f40424dd-b515-4029-b062-02fbae312213-istio-podinfo\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.058761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058684    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-socket\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.058761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058735    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-envoy\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.058941 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058770    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-credential-socket\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.058941 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058826    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-istio-token\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-istio-token\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.058941 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058884    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-kube-api-access-hnwf2\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-kube-api-access-hnwf2\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.058941 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.058928    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-data\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.059139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059004    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-certs\") pod \"f40424dd-b515-4029-b062-02fbae312213\" (UID: \"f40424dd-b515-4029-b062-02fbae312213\") "
Oct 30 22:21:46.059139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059086    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-socket" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:46.059139 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059109    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f40424dd-b515-4029-b062-02fbae312213-istiod-ca-cert" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:46.059291 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059229    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-credential-socket" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:46.059443 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059421    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-data" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:46.059507 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059449    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f40424dd-b515-4029-b062-02fbae312213-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.059507 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059470    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.059507 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059483    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.059892 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.059869    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-certs" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:46.061278 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.061241    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-kube-api-access-hnwf2" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "kube-api-access-hnwf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:46.061448 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.061371    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f40424dd-b515-4029-b062-02fbae312213-istio-podinfo" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:46.061508 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.061451    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-envoy" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:46.061652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.061631    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-istio-token" pod "f40424dd-b515-4029-b062-02fbae312213" (UID: "f40424dd-b515-4029-b062-02fbae312213"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:46.160273 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.160191    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f40424dd-b515-4029-b062-02fbae312213-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.160273 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.160230    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.160273 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.160250    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.160273 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.160264    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnwf2\" (UniqueName: \"kubernetes.io/projected/f40424dd-b515-4029-b062-02fbae312213-kube-api-access-hnwf2\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.160523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.160279    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.160523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.160288    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f40424dd-b515-4029-b062-02fbae312213-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.285761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.285735    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 22:21:46.362630 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.362575    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7d7cc94-b718-4145-a7a1-c7c91876d78e-kube-api-access-wsc5x\" (UniqueName: \"kubernetes.io/projected/e7d7cc94-b718-4145-a7a1-c7c91876d78e-kube-api-access-wsc5x\") pod \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") "
Oct 30 22:21:46.362718 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.362655    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-metrics-certs\") pod \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") "
Oct 30 22:21:46.362773 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.362735    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-webhook-cert\") pod \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\" (UID: \"e7d7cc94-b718-4145-a7a1-c7c91876d78e\") "
Oct 30 22:21:46.364699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.364668    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d7cc94-b718-4145-a7a1-c7c91876d78e-kube-api-access-wsc5x" pod "e7d7cc94-b718-4145-a7a1-c7c91876d78e" (UID: "e7d7cc94-b718-4145-a7a1-c7c91876d78e"). InnerVolumeSpecName "kube-api-access-wsc5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:46.364801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.364693    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-metrics-certs" pod "e7d7cc94-b718-4145-a7a1-c7c91876d78e" (UID: "e7d7cc94-b718-4145-a7a1-c7c91876d78e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:46.364801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.364708    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-webhook-cert" pod "e7d7cc94-b718-4145-a7a1-c7c91876d78e" (UID: "e7d7cc94-b718-4145-a7a1-c7c91876d78e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:46.463636 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.463609    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.463636 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.463633    5561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d7cc94-b718-4145-a7a1-c7c91876d78e-webhook-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.463818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.463644    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsc5x\" (UniqueName: \"kubernetes.io/projected/e7d7cc94-b718-4145-a7a1-c7c91876d78e-kube-api-access-wsc5x\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:46.895147 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.895119    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"
Oct 30 22:21:46.895147 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.895123    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt" event={"ID":"f40424dd-b515-4029-b062-02fbae312213","Type":"ContainerDied","Data":"6bbb44066a72bca773a60f55825a03d4cab2692cc0aebc67409b8f74bd7e5207"}
Oct 30 22:21:46.895738 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.895184    5561 scope.go:122] "RemoveContainer" containerID="e9c432a74870040f54a270481dd4e041a93941f747034aa8c3afe149e512f357"
Oct 30 22:21:46.896880 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.896852    5561 generic.go:356] "Generic (PLEG): container finished" podID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerID="37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e" exitCode=0
Oct 30 22:21:46.896998 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.896892    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-l7ph6"
Oct 30 22:21:46.896998 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.896928    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-l7ph6" event={"ID":"e7d7cc94-b718-4145-a7a1-c7c91876d78e","Type":"ContainerDied","Data":"37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e"}
Oct 30 22:21:46.896998 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.896976    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-l7ph6" event={"ID":"e7d7cc94-b718-4145-a7a1-c7c91876d78e","Type":"ContainerDied","Data":"b2981565dc961d01c85bf06412a1c3592ef8bc01501c530ba77f98b463becd54"}
Oct 30 22:21:46.904790 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.904694    5561 scope.go:122] "RemoveContainer" containerID="37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e"
Oct 30 22:21:46.913285 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.913267    5561 scope.go:122] "RemoveContainer" containerID="3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4"
Oct 30 22:21:46.920742 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.920724    5561 scope.go:122] "RemoveContainer" containerID="37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e"
Oct 30 22:21:46.920986 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:21:46.920969    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e\": container with ID starting with 37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e not found: ID does not exist" containerID="37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e"
Oct 30 22:21:46.921026 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.920994    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e"} err="failed to get container status \"37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e\": rpc error: code = NotFound desc = could not find container \"37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e\": container with ID starting with 37ca1d65359a04b544b8ce3e487c0f19c678f1f5dcf90229ca85aa6aa70ad83e not found: ID does not exist"
Oct 30 22:21:46.921026 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.921012    5561 scope.go:122] "RemoveContainer" containerID="3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4"
Oct 30 22:21:46.921235 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:21:46.921215    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4\": container with ID starting with 3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4 not found: ID does not exist" containerID="3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4"
Oct 30 22:21:46.921274 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:46.921242    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4"} err="failed to get container status \"3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4\": rpc error: code = NotFound desc = could not find container \"3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4\": container with ID starting with 3553ffbd5cbce20d3c1417b701c95aa5a4a90173389d45d42c8de6f4706567c4 not found: ID does not exist"
Oct 30 22:21:47.062120 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:47.062053    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"]
Oct 30 22:21:47.088170 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:47.088140    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt"]
Oct 30 22:21:47.178746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:47.178723    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7567487b8b-l7ph6"]
Oct 30 22:21:47.182916 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:47.182898    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7567487b8b-l7ph6"]
Oct 30 22:21:47.844816 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:47.844783    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" path="/var/lib/kubelet/pods/e7d7cc94-b718-4145-a7a1-c7c91876d78e/volumes"
Oct 30 22:21:47.845261 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:47.845244    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f40424dd-b515-4029-b062-02fbae312213" path="/var/lib/kubelet/pods/f40424dd-b515-4029-b062-02fbae312213/volumes"
Oct 30 22:21:47.875811 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:21:47.875782    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:21:52.979081 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:21:52.979041    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:22:12.945773 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:12.945739    5561 scope.go:122] "RemoveContainer" containerID="8c23e2d22f0c8ddecfdb664e1e4f22d982b151b7aca7e63e01137ad64108e6f2"
Oct 30 22:22:15.159394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:15.159361    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:22:22.981445 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:22:22.981341    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:22:25.715462 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715431    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7567487b8b-trv6g"]
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715786    5561 cpu_manager.go:414] "containerMap: removing container" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerName="manager"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715797    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerName="manager"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715813    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a840bba1-a641-441b-966d-772cb050cb0c" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715818    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a840bba1-a641-441b-966d-772cb050cb0c" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715828    5561 cpu_manager.go:414] "containerMap: removing container" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerName="manager"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715833    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerName="manager"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715840    5561 cpu_manager.go:414] "containerMap: removing container" podUID="4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" containerName="maas-api"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715845    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" containerName="maas-api"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715852    5561 cpu_manager.go:414] "containerMap: removing container" podUID="c9780746-8d6a-4e0f-a91d-f14f18d68629" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715857    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9780746-8d6a-4e0f-a91d-f14f18d68629" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715865    5561 cpu_manager.go:414] "containerMap: removing container" podUID="4db227fc-374f-45bd-aa45-4c5abbe45e74" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715871    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4db227fc-374f-45bd-aa45-4c5abbe45e74" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715877    5561 cpu_manager.go:414] "containerMap: removing container" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715882    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715889    5561 cpu_manager.go:414] "containerMap: removing container" podUID="e62f24f9-d580-474f-a109-ff1c9e879bcf" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715894    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e62f24f9-d580-474f-a109-ff1c9e879bcf" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715905    5561 cpu_manager.go:414] "containerMap: removing container" podUID="55852795-178a-4281-b268-eff4af7382d2" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715910    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55852795-178a-4281-b268-eff4af7382d2" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715916    5561 cpu_manager.go:414] "containerMap: removing container" podUID="6a0c8114-242f-4cdd-9a38-eff32ed82211" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715921    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a0c8114-242f-4cdd-9a38-eff32ed82211" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715928    5561 cpu_manager.go:414] "containerMap: removing container" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715932    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715939    5561 cpu_manager.go:414] "containerMap: removing container" podUID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715943    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715950    5561 cpu_manager.go:414] "containerMap: removing container" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715954    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715969    5561 cpu_manager.go:414] "containerMap: removing container" podUID="337476fb-9958-4043-abbc-6e36bb5da051" containerName="revoke-keys"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715974    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="337476fb-9958-4043-abbc-6e36bb5da051" containerName="revoke-keys"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715981    5561 cpu_manager.go:414] "containerMap: removing container" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715986    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715993    5561 cpu_manager.go:414] "containerMap: removing container" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.715998    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716062    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerName="manager"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716072    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a840bba1-a641-441b-966d-772cb050cb0c" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716078    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7d7cc94-b718-4145-a7a1-c7c91876d78e" containerName="manager"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716085    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cad39ee-9e75-4c0a-90bf-2f9e4c1fe3e2" containerName="maas-api"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716091    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9780746-8d6a-4e0f-a91d-f14f18d68629" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716097    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="4db227fc-374f-45bd-aa45-4c5abbe45e74" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716101    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716106    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="e62f24f9-d580-474f-a109-ff1c9e879bcf" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716112    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="55852795-178a-4281-b268-eff4af7382d2" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716118    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a0c8114-242f-4cdd-9a38-eff32ed82211" containerName="payload-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716124    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bd4292f-6a46-4aa4-9166-1637614c68f0" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716129    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="9307b981-13a4-4e6a-836d-e9d0911502ba" containerName="main"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716134    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdf44047-bb4f-4860-a861-a2c5bfd4358b" containerName="payload-pre-processing"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716141    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="337476fb-9958-4043-abbc-6e36bb5da051" containerName="revoke-keys"
Oct 30 22:22:25.716174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.716145    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="f40424dd-b515-4029-b062-02fbae312213" containerName="istio-proxy"
Oct 30 22:22:25.720821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.720804    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.824762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.824729    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:22:25.824903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.824801    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:22:25.824903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.824805    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7q82b\""
Oct 30 22:22:25.824903 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.824839    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:22:25.826496 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.826476    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7567487b8b-trv6g"]
Oct 30 22:22:25.826769 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.826750    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-webhook-cert\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.826813 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.826795    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27cr\" (UniqueName: \"kubernetes.io/projected/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-kube-api-access-k27cr\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.826885 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.826867    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-metrics-certs\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.928083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.928049    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27cr\" (UniqueName: \"kubernetes.io/projected/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-kube-api-access-k27cr\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.928231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.928096    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-metrics-certs\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.928231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.928164    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-webhook-cert\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.930689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.930663    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-webhook-cert\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.930847 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.930751    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-metrics-certs\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:25.985777 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:25.985720    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27cr\" (UniqueName: \"kubernetes.io/projected/2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1-kube-api-access-k27cr\") pod \"maas-controller-7567487b8b-trv6g\" (UID: \"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1\") " pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:26.033469 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:26.033446    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:26.406227 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:26.406192    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7567487b8b-trv6g"]
Oct 30 22:22:26.407817 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:22:26.407788    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2980b3fe_2c79_46a2_a8e9_11fe52d2b3e1.slice/crio-1a7bbf4963ea42f40d4587b91a35db1710425576b0be4c209ae095aa7d939495 WatchSource:0}: Error finding container 1a7bbf4963ea42f40d4587b91a35db1710425576b0be4c209ae095aa7d939495: Status 404 returned error can't find the container with id 1a7bbf4963ea42f40d4587b91a35db1710425576b0be4c209ae095aa7d939495
Oct 30 22:22:26.409961 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:26.409940    5561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:22:27.102177 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:27.102140    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-trv6g" event={"ID":"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1","Type":"ContainerStarted","Data":"1a7bbf4963ea42f40d4587b91a35db1710425576b0be4c209ae095aa7d939495"}
Oct 30 22:22:28.110756 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:28.110700    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-trv6g" event={"ID":"2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1","Type":"ContainerStarted","Data":"503e02a23d5945420db2fd43d0bfa89f5aaf9cfa0153f63e4f475413788504b1"}
Oct 30 22:22:28.111241 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:28.111199    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:28.229626 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:28.229569    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7567487b8b-trv6g" podStartSLOduration=2.453740246 podStartE2EDuration="3.22955637s" totalImagesPullingTime="775.816124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:28.228151525 +0000 UTC m=+2896.505844997" watchObservedRunningTime="2026-10-30 22:22:28.22955637 +0000 UTC m=+2896.507249842"
Oct 30 22:22:39.118052 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:39.118022    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7567487b8b-trv6g"
Oct 30 22:22:52.980113 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:22:52.980068    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:22:59.044994 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:59.044960    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"]
Oct 30 22:22:59.096178 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:59.096140    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-15004b6e-hjzlm"]
Oct 30 22:22:59.843843 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:22:59.843806    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e200a26-a686-495a-877a-1676eaaee331" path="/var/lib/kubelet/pods/7e200a26-a686-495a-877a-1676eaaee331/volumes"
Oct 30 22:23:10.988880 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:10.988846    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:23:12.046369 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:12.046315    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"]
Oct 30 22:23:12.097283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:12.097222    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c97b9e0b-pknqq"]
Oct 30 22:23:13.336490 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:13.336458    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:23:13.336883 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:13.336521    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:23:13.844175 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:13.844135    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="337476fb-9958-4043-abbc-6e36bb5da051" path="/var/lib/kubelet/pods/337476fb-9958-4043-abbc-6e36bb5da051/volumes"
Oct 30 22:23:22.980868 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:23:22.980831    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:23:42.450119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:42.450080    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:23:52.085171 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:23:52.085086    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:23:52.980269 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:23:52.980227    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:24:11.151558 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:24:11.151527    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:24:12.395993 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:24:12.395966    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:24:12.427244 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:24:12.427221    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:24:13.005745 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:24:13.005716    5561 scope.go:122] "RemoveContainer" containerID="ec4d98fdebd6a34372fe6ee9e34e379b7a10dc8dd8d0f64fdf2b4bb70ee083d8"
Oct 30 22:24:13.014362 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:24:13.014330    5561 scope.go:122] "RemoveContainer" containerID="6d79047d581862520af82fdc37a8afe20dceea93ead49610c43c631ab794e7f4"
Oct 30 22:24:22.979316 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:24:22.979274    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:24:27.974824 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:24:27.974794    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:24:52.980316 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:24:52.980276    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:25:19.066186 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:25:19.066145    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:25:22.980631 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:25:22.980550    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:25:29.751960 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:25:29.751925    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:25:40.644680 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:25:40.644638    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:25:52.980186 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:25:52.980146    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:26:04.567274 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:26:04.567238    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:26:13.897179 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:26:13.897143    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:26:22.981042 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:26:22.980998    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:26:42.677517 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:26:42.677478    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:26:52.980937 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:26:52.980848    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:26:59.874468 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:26:59.874433    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:27:21.872410 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:27:21.872371    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:27:22.987383 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:27:22.987315    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:27:52.979156 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:27:52.979117    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:28:00.951712 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:28:00.951679    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:28:12.271550 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:28:12.271512    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:28:22.979985 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:28:22.979894    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:28:23.556917 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:28:23.556881    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:28:52.979853 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:28:52.979813    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:28:58.340505 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:28:58.340474    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:29:12.423199 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:29:12.423112    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:29:12.455233 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:29:12.455210    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:29:22.981301 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:29:22.981266    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:29:26.961980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:29:26.961949    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:29:37.642846 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:29:37.642810    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:29:52.981027 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:29:52.980948    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:30:16.974441 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:16.974403    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"]
Oct 30 22:30:16.975165 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:16.974917    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="fef9544f-6a1c-4289-997c-532d42ba7d38" containerName="istio-proxy"
Oct 30 22:30:16.977886 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:16.977869    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.038672 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.038645    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"]
Oct 30 22:30:17.086940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.086916    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-5b7354\""
Oct 30 22:30:17.102924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.102890    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"]
Oct 30 22:30:17.111188 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.111153    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.145604 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.145563    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhrfn\" (UniqueName: \"kubernetes.io/projected/22fbba2f-0907-4f63-bd95-98838a2802b3-kube-api-access-mhrfn\") pod \"payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.145804 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.145664    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fbba2f-0907-4f63-bd95-98838a2802b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.173828 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.173794    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"]
Oct 30 22:30:17.174017 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.173895    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-1c9e57a1-dockercfg-7qqxf\""
Oct 30 22:30:17.246455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.246371    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhrfn\" (UniqueName: \"kubernetes.io/projected/22fbba2f-0907-4f63-bd95-98838a2802b3-kube-api-access-mhrfn\") pod \"payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.246612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.246455    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fbba2f-0907-4f63-bd95-98838a2802b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.246612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.246510    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkhw6\" (UniqueName: \"kubernetes.io/projected/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-kube-api-access-rkhw6\") pod \"payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.246612 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.246565    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.247153 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.247131    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fbba2f-0907-4f63-bd95-98838a2802b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.305621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.305599    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhrfn\" (UniqueName: \"kubernetes.io/projected/22fbba2f-0907-4f63-bd95-98838a2802b3-kube-api-access-mhrfn\") pod \"payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.347715 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.347687    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkhw6\" (UniqueName: \"kubernetes.io/projected/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-kube-api-access-rkhw6\") pod \"payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.347845 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.347729    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.348411 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.348391    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.413882 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.413857    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkhw6\" (UniqueName: \"kubernetes.io/projected/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-kube-api-access-rkhw6\") pod \"payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.431486 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.431463    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:17.590493 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.590418    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:17.808124 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.808100    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"]
Oct 30 22:30:17.810037 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:30:17.810008    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a78e1fa_9cd7_49fb_a2ce_98c919db1104.slice/crio-2eb9ce3229f81d97ca82198b340a220920728df7660f693c731ea859fb230b54 WatchSource:0}: Error finding container 2eb9ce3229f81d97ca82198b340a220920728df7660f693c731ea859fb230b54: Status 404 returned error can't find the container with id 2eb9ce3229f81d97ca82198b340a220920728df7660f693c731ea859fb230b54
Oct 30 22:30:17.966069 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:17.966044    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"]
Oct 30 22:30:17.968232 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:30:17.968208    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22fbba2f_0907_4f63_bd95_98838a2802b3.slice/crio-ca1f99a14d54f7c9cb26f112fe7e850c9bd65e9ae7d297f060833a0890ceef5a WatchSource:0}: Error finding container ca1f99a14d54f7c9cb26f112fe7e850c9bd65e9ae7d297f060833a0890ceef5a: Status 404 returned error can't find the container with id ca1f99a14d54f7c9cb26f112fe7e850c9bd65e9ae7d297f060833a0890ceef5a
Oct 30 22:30:18.455101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:18.455061    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" event={"ID":"2a78e1fa-9cd7-49fb-a2ce-98c919db1104","Type":"ContainerStarted","Data":"c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3"}
Oct 30 22:30:18.455563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:18.455125    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" event={"ID":"2a78e1fa-9cd7-49fb-a2ce-98c919db1104","Type":"ContainerStarted","Data":"2eb9ce3229f81d97ca82198b340a220920728df7660f693c731ea859fb230b54"}
Oct 30 22:30:18.455563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:18.455154    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:18.456552 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:18.456521    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" event={"ID":"22fbba2f-0907-4f63-bd95-98838a2802b3","Type":"ContainerStarted","Data":"0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3"}
Oct 30 22:30:18.456649 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:18.456564    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" event={"ID":"22fbba2f-0907-4f63-bd95-98838a2802b3","Type":"ContainerStarted","Data":"ca1f99a14d54f7c9cb26f112fe7e850c9bd65e9ae7d297f060833a0890ceef5a"}
Oct 30 22:30:18.456709 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:18.456664    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:18.700784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:18.697181    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" podStartSLOduration=1.6971588770000001 podStartE2EDuration="1.697158877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:18.575732307 +0000 UTC m=+3366.853425778" watchObservedRunningTime="2026-10-30 22:30:18.697158877 +0000 UTC m=+3366.974852350"
Oct 30 22:30:19.607892 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.607840    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" podStartSLOduration=3.607826971 podStartE2EDuration="3.607826971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:18.700650708 +0000 UTC m=+3366.978344180" watchObservedRunningTime="2026-10-30 22:30:19.607826971 +0000 UTC m=+3367.885520478"
Oct 30 22:30:19.610104 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.608092    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"]
Oct 30 22:30:19.611236 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.611220    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.665429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.665405    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-5b7354-serving-cert\""
Oct 30 22:30:19.717298 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.717269    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"]
Oct 30 22:30:19.770869 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.770842    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74bece9f-ca59-4695-bd00-3a5844f87794-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.771004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.770963    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.771043 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.771031    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhm4b\" (UniqueName: \"kubernetes.io/projected/74bece9f-ca59-4695-bd00-3a5844f87794-kube-api-access-rhm4b\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.771084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.771049    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.871702 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.871678    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.871827 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.871741    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhm4b\" (UniqueName: \"kubernetes.io/projected/74bece9f-ca59-4695-bd00-3a5844f87794-kube-api-access-rhm4b\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.871827 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.871773    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.871827 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.871811    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74bece9f-ca59-4695-bd00-3a5844f87794-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.872511 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.872488    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74bece9f-ca59-4695-bd00-3a5844f87794-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.874206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.874184    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.874278 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.874262    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:19.931761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:19.931737    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhm4b\" (UniqueName: \"kubernetes.io/projected/74bece9f-ca59-4695-bd00-3a5844f87794-kube-api-access-rhm4b\") pod \"maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:20.224083 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:20.224002    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:20.596243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:20.596195    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"]
Oct 30 22:30:20.598478 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:30:20.598450    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74bece9f_ca59_4695_bd00_3a5844f87794.slice/crio-d8f24c20b67bd75856b6888f0657d131465f51ce4b22738a0dd3d690f6a963a1 WatchSource:0}: Error finding container d8f24c20b67bd75856b6888f0657d131465f51ce4b22738a0dd3d690f6a963a1: Status 404 returned error can't find the container with id d8f24c20b67bd75856b6888f0657d131465f51ce4b22738a0dd3d690f6a963a1
Oct 30 22:30:20.599805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:20.599786    5561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:30:21.472406 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:21.472371    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q" event={"ID":"74bece9f-ca59-4695-bd00-3a5844f87794","Type":"ContainerStarted","Data":"d8f24c20b67bd75856b6888f0657d131465f51ce4b22738a0dd3d690f6a963a1"}
Oct 30 22:30:22.477684 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:22.477649    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q" event={"ID":"74bece9f-ca59-4695-bd00-3a5844f87794","Type":"ContainerStarted","Data":"3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d"}
Oct 30 22:30:22.478095 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:22.477774    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:22.600665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:22.600615    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q" podStartSLOduration=2.640377045 podStartE2EDuration="3.600601431s" totalImagesPullingTime="960.224386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:22.599041185 +0000 UTC m=+3370.876734658" watchObservedRunningTime="2026-10-30 22:30:22.600601431 +0000 UTC m=+3370.878294901"
Oct 30 22:30:22.979635 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:30:22.979597    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:30:27.790227 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:27.790197    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:30:28.488449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:28.488419    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:30:29.465202 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:29.465173    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:30:29.465638 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:29.465446    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:30:47.484122 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:30:47.484086    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:30:52.979822 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:30:52.979780    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:31:12.171175 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:12.171135    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:31:22.980315 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:31:22.980235    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:31:41.488267 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:41.488237    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:31:49.100995 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:49.100961    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"]
Oct 30 22:31:49.205683 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:49.205652    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"]
Oct 30 22:31:52.980176 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:31:52.980139    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:31:54.101711 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:54.101663    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" podUID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" containerName="payload-processing" containerID="cri-o://c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3" gracePeriod=25
Oct 30 22:31:54.206063 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:54.206027    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" podUID="22fbba2f-0907-4f63-bd95-98838a2802b3" containerName="payload-pre-processing" containerID="cri-o://0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3" gracePeriod=25
Oct 30 22:31:57.331022 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:57.330987    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"]
Oct 30 22:31:57.331473 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:57.331185    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q" podUID="74bece9f-ca59-4695-bd00-3a5844f87794" containerName="maas-api" containerID="cri-o://3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d" gracePeriod=30
Oct 30 22:31:58.484056 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.484017    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q" podUID="74bece9f-ca59-4695-bd00-3a5844f87794" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.130:8443/health\": dial tcp 10.133.0.130:8443: connect: connection refused"
Oct 30 22:31:58.585971 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.585944    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:31:58.681691 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.681602    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-maas-api-tls\") pod \"74bece9f-ca59-4695-bd00-3a5844f87794\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") "
Oct 30 22:31:58.681691 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.681679    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74bece9f-ca59-4695-bd00-3a5844f87794-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74bece9f-ca59-4695-bd00-3a5844f87794-openshift-service-ca\") pod \"74bece9f-ca59-4695-bd00-3a5844f87794\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") "
Oct 30 22:31:58.681940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.681708    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74bece9f-ca59-4695-bd00-3a5844f87794-kube-api-access-rhm4b\" (UniqueName: \"kubernetes.io/projected/74bece9f-ca59-4695-bd00-3a5844f87794-kube-api-access-rhm4b\") pod \"74bece9f-ca59-4695-bd00-3a5844f87794\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") "
Oct 30 22:31:58.681940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.681736    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-metrics-certs\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-metrics-certs\") pod \"74bece9f-ca59-4695-bd00-3a5844f87794\" (UID: \"74bece9f-ca59-4695-bd00-3a5844f87794\") "
Oct 30 22:31:58.682106 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.682079    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74bece9f-ca59-4695-bd00-3a5844f87794-openshift-service-ca" pod "74bece9f-ca59-4695-bd00-3a5844f87794" (UID: "74bece9f-ca59-4695-bd00-3a5844f87794"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:58.682205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.682190    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74bece9f-ca59-4695-bd00-3a5844f87794-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.683967 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.683939    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-metrics-certs" pod "74bece9f-ca59-4695-bd00-3a5844f87794" (UID: "74bece9f-ca59-4695-bd00-3a5844f87794"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:58.684080 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.684035    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74bece9f-ca59-4695-bd00-3a5844f87794-kube-api-access-rhm4b" pod "74bece9f-ca59-4695-bd00-3a5844f87794" (UID: "74bece9f-ca59-4695-bd00-3a5844f87794"). InnerVolumeSpecName "kube-api-access-rhm4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:58.684237 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.684209    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-maas-api-tls" pod "74bece9f-ca59-4695-bd00-3a5844f87794" (UID: "74bece9f-ca59-4695-bd00-3a5844f87794"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:58.783025 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.782989    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhm4b\" (UniqueName: \"kubernetes.io/projected/74bece9f-ca59-4695-bd00-3a5844f87794-kube-api-access-rhm4b\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.783025 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.783017    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.783025 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.783028    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74bece9f-ca59-4695-bd00-3a5844f87794-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.916828 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.916788    5561 generic.go:356] "Generic (PLEG): container finished" podID="74bece9f-ca59-4695-bd00-3a5844f87794" containerID="3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d" exitCode=0
Oct 30 22:31:58.917014 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.916838    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"
Oct 30 22:31:58.917014 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.916854    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q" event={"ID":"74bece9f-ca59-4695-bd00-3a5844f87794","Type":"ContainerDied","Data":"3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d"}
Oct 30 22:31:58.917014 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.916903    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q" event={"ID":"74bece9f-ca59-4695-bd00-3a5844f87794","Type":"ContainerDied","Data":"d8f24c20b67bd75856b6888f0657d131465f51ce4b22738a0dd3d690f6a963a1"}
Oct 30 22:31:58.917014 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.916926    5561 scope.go:122] "RemoveContainer" containerID="3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d"
Oct 30 22:31:58.926166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.926145    5561 scope.go:122] "RemoveContainer" containerID="3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d"
Oct 30 22:31:58.926430 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:31:58.926410    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d\": container with ID starting with 3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d not found: ID does not exist" containerID="3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d"
Oct 30 22:31:58.926495 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:58.926439    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d"} err="failed to get container status \"3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d\": rpc error: code = NotFound desc = could not find container \"3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d\": container with ID starting with 3b06dac4bd65438093184ce8ce80551dfdc47945a389838f0095bdfa5e05912d not found: ID does not exist"
Oct 30 22:31:59.081772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:59.081692    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"]
Oct 30 22:31:59.120877 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:59.120846    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5b7354-55dd8755b6-8dw7q"]
Oct 30 22:31:59.463659 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:59.463615    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" podUID="22fbba2f-0907-4f63-bd95-98838a2802b3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:31:59.463814 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:59.463658    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" podUID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:31:59.844932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:31:59.844848    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74bece9f-ca59-4695-bd00-3a5844f87794" path="/var/lib/kubelet/pods/74bece9f-ca59-4695-bd00-3a5844f87794/volumes"
Oct 30 22:32:02.185483 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:02.185450    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:32:09.463973 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:09.463930    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" podUID="22fbba2f-0907-4f63-bd95-98838a2802b3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:32:09.464562 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:09.463940    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" podUID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:32:14.350125 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.350094    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:32:14.433065 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.432935    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-plugins-config-volume\") pod \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") "
Oct 30 22:32:14.433065 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.433018    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-kube-api-access-rkhw6\" (UniqueName: \"kubernetes.io/projected/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-kube-api-access-rkhw6\") pod \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\" (UID: \"2a78e1fa-9cd7-49fb-a2ce-98c919db1104\") "
Oct 30 22:32:14.433345 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.433319    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-plugins-config-volume" pod "2a78e1fa-9cd7-49fb-a2ce-98c919db1104" (UID: "2a78e1fa-9cd7-49fb-a2ce-98c919db1104"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:32:14.435118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.435092    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-kube-api-access-rkhw6" pod "2a78e1fa-9cd7-49fb-a2ce-98c919db1104" (UID: "2a78e1fa-9cd7-49fb-a2ce-98c919db1104"). InnerVolumeSpecName "kube-api-access-rkhw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:14.442913 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.442894    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:32:14.534427 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.534397    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22fbba2f-0907-4f63-bd95-98838a2802b3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fbba2f-0907-4f63-bd95-98838a2802b3-plugins-config-volume\") pod \"22fbba2f-0907-4f63-bd95-98838a2802b3\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") "
Oct 30 22:32:14.534567 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.534515    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22fbba2f-0907-4f63-bd95-98838a2802b3-kube-api-access-mhrfn\" (UniqueName: \"kubernetes.io/projected/22fbba2f-0907-4f63-bd95-98838a2802b3-kube-api-access-mhrfn\") pod \"22fbba2f-0907-4f63-bd95-98838a2802b3\" (UID: \"22fbba2f-0907-4f63-bd95-98838a2802b3\") "
Oct 30 22:32:14.534737 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.534715    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22fbba2f-0907-4f63-bd95-98838a2802b3-plugins-config-volume" pod "22fbba2f-0907-4f63-bd95-98838a2802b3" (UID: "22fbba2f-0907-4f63-bd95-98838a2802b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:32:14.534803 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.534782    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:14.534855 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.534803    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkhw6\" (UniqueName: \"kubernetes.io/projected/2a78e1fa-9cd7-49fb-a2ce-98c919db1104-kube-api-access-rkhw6\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:14.536433 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.536414    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22fbba2f-0907-4f63-bd95-98838a2802b3-kube-api-access-mhrfn" pod "22fbba2f-0907-4f63-bd95-98838a2802b3" (UID: "22fbba2f-0907-4f63-bd95-98838a2802b3"). InnerVolumeSpecName "kube-api-access-mhrfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:14.635868 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.635794    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fbba2f-0907-4f63-bd95-98838a2802b3-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:14.635868 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.635823    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhrfn\" (UniqueName: \"kubernetes.io/projected/22fbba2f-0907-4f63-bd95-98838a2802b3-kube-api-access-mhrfn\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:14.990344 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.990264    5561 generic.go:356] "Generic (PLEG): container finished" podID="22fbba2f-0907-4f63-bd95-98838a2802b3" containerID="0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3" exitCode=0
Oct 30 22:32:14.990344 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.990311    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"
Oct 30 22:32:14.990560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.990343    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" event={"ID":"22fbba2f-0907-4f63-bd95-98838a2802b3","Type":"ContainerDied","Data":"0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3"}
Oct 30 22:32:14.990560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.990410    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk" event={"ID":"22fbba2f-0907-4f63-bd95-98838a2802b3","Type":"ContainerDied","Data":"ca1f99a14d54f7c9cb26f112fe7e850c9bd65e9ae7d297f060833a0890ceef5a"}
Oct 30 22:32:14.990560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.990441    5561 scope.go:122] "RemoveContainer" containerID="0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3"
Oct 30 22:32:14.991750 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.991727    5561 generic.go:356] "Generic (PLEG): container finished" podID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" containerID="c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3" exitCode=0
Oct 30 22:32:14.991875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.991773    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" event={"ID":"2a78e1fa-9cd7-49fb-a2ce-98c919db1104","Type":"ContainerDied","Data":"c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3"}
Oct 30 22:32:14.991875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.991786    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"
Oct 30 22:32:14.991875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:14.991810    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp" event={"ID":"2a78e1fa-9cd7-49fb-a2ce-98c919db1104","Type":"ContainerDied","Data":"2eb9ce3229f81d97ca82198b340a220920728df7660f693c731ea859fb230b54"}
Oct 30 22:32:15.000148 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.000123    5561 scope.go:122] "RemoveContainer" containerID="0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3"
Oct 30 22:32:15.000460 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:32:15.000434    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3\": container with ID starting with 0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3 not found: ID does not exist" containerID="0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3"
Oct 30 22:32:15.000533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.000471    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3"} err="failed to get container status \"0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3\": rpc error: code = NotFound desc = could not find container \"0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3\": container with ID starting with 0aaf4630093b5a75e5006aaba77bab9f82af40a3b00f0e36cfc16ee981cbffa3 not found: ID does not exist"
Oct 30 22:32:15.000533 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.000496    5561 scope.go:122] "RemoveContainer" containerID="c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3"
Oct 30 22:32:15.008192 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.008177    5561 scope.go:122] "RemoveContainer" containerID="c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3"
Oct 30 22:32:15.008474 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:32:15.008454    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3\": container with ID starting with c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3 not found: ID does not exist" containerID="c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3"
Oct 30 22:32:15.008535 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.008486    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3"} err="failed to get container status \"c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3\": rpc error: code = NotFound desc = could not find container \"c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3\": container with ID starting with c450a16b61b18cf529032b7848e1e6fa8e9a25f60d3ec23b61474932c8fc07f3 not found: ID does not exist"
Oct 30 22:32:15.153966 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.153944    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"]
Oct 30 22:32:15.157772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.157752    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp"]
Oct 30 22:32:15.266128 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.266077    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"]
Oct 30 22:32:15.271522 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.271503    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk"]
Oct 30 22:32:15.844004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.843975    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22fbba2f-0907-4f63-bd95-98838a2802b3" path="/var/lib/kubelet/pods/22fbba2f-0907-4f63-bd95-98838a2802b3/volumes"
Oct 30 22:32:15.844389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:32:15.844374    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" path="/var/lib/kubelet/pods/2a78e1fa-9cd7-49fb-a2ce-98c919db1104/volumes"
Oct 30 22:32:22.980217 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:32:22.980173    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:32:52.982003 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:32:52.981918    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:33:22.980137 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:33:22.980100    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:33:52.982539 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:33:52.982491    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:34:01.355317 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:01.355281    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:34:12.452123 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:12.452098    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:34:12.484855 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:12.484834    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:34:22.980852 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:34:22.980767    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:34:25.262619 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:25.262588    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:34:33.466847 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:33.466811    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:34:52.979046 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:34:52.979004    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:34:54.382015 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.381980    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:34:54.700471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700398    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"]
Oct 30 22:34:54.700795 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700781    5561 cpu_manager.go:414] "containerMap: removing container" podUID="74bece9f-ca59-4695-bd00-3a5844f87794" containerName="maas-api"
Oct 30 22:34:54.700848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700795    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74bece9f-ca59-4695-bd00-3a5844f87794" containerName="maas-api"
Oct 30 22:34:54.700848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700808    5561 cpu_manager.go:414] "containerMap: removing container" podUID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" containerName="payload-processing"
Oct 30 22:34:54.700848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700814    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" containerName="payload-processing"
Oct 30 22:34:54.700848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700829    5561 cpu_manager.go:414] "containerMap: removing container" podUID="22fbba2f-0907-4f63-bd95-98838a2802b3" containerName="payload-pre-processing"
Oct 30 22:34:54.700848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700834    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22fbba2f-0907-4f63-bd95-98838a2802b3" containerName="payload-pre-processing"
Oct 30 22:34:54.701004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700886    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="74bece9f-ca59-4695-bd00-3a5844f87794" containerName="maas-api"
Oct 30 22:34:54.701004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700893    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a78e1fa-9cd7-49fb-a2ce-98c919db1104" containerName="payload-processing"
Oct 30 22:34:54.701004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.700902    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="22fbba2f-0907-4f63-bd95-98838a2802b3" containerName="payload-pre-processing"
Oct 30 22:34:54.703921 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.703899    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.766381 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.765806    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-aafbca-openshift-default-dockercfg-xn68h\""
Oct 30 22:34:54.783949 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.783907    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-token\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784157 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784003    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-data\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784157 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784041    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-certs\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784157 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784100    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-podinfo\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784157 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784126    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-socket\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784157 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784152    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4712d6-9a59-4e7f-a07e-c97192c73465-istiod-ca-cert\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784467 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784216    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-credential-socket\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784467 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784248    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvljl\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-kube-api-access-fvljl\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.784467 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.784308    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-envoy\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.818806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.818772    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"]
Oct 30 22:34:54.885054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885009    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvljl\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-kube-api-access-fvljl\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885264 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885094    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-envoy\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885264 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885131    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-token\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885264 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885201    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-data\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885264 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885234    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-certs\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885555 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885413    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-podinfo\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885555 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885446    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-socket\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885555 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885471    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4712d6-9a59-4e7f-a07e-c97192c73465-istiod-ca-cert\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.885555 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.885539    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-credential-socket\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.888699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.887277    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4712d6-9a59-4e7f-a07e-c97192c73465-istiod-ca-cert\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.888699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.887841    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-socket\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.888699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.887924    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-certs\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.888699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.888104    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-data\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.888699 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.888243    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-credential-socket\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.889066 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.888899    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-podinfo\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.890224 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.890198    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-envoy\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.950320 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.950281    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvljl\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-kube-api-access-fvljl\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:54.961690 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:54.961624    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-token\") pod \"e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") " pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:55.022140 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.021542    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:55.447815 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.447787    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"]
Oct 30 22:34:55.449111 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:34:55.449080    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4712d6_9a59_4e7f_a07e_c97192c73465.slice/crio-f0ca5b47a9f4589292695ac92aa2416620c5d98e40343ec026e6ada29f08e5f2 WatchSource:0}: Error finding container f0ca5b47a9f4589292695ac92aa2416620c5d98e40343ec026e6ada29f08e5f2: Status 404 returned error can't find the container with id f0ca5b47a9f4589292695ac92aa2416620c5d98e40343ec026e6ada29f08e5f2
Oct 30 22:34:55.451277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.451243    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:34:55.451340 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.451330    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:34:55.451446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.451376    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:34:55.733202 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.733127    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" event={"ID":"8c4712d6-9a59-4e7f-a07e-c97192c73465","Type":"ContainerStarted","Data":"43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de"}
Oct 30 22:34:55.733202 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.733177    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" event={"ID":"8c4712d6-9a59-4e7f-a07e-c97192c73465","Type":"ContainerStarted","Data":"f0ca5b47a9f4589292695ac92aa2416620c5d98e40343ec026e6ada29f08e5f2"}
Oct 30 22:34:55.855164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:55.855107    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" podStartSLOduration=1.85509309 podStartE2EDuration="1.85509309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:55.853771028 +0000 UTC m=+3644.131464501" watchObservedRunningTime="2026-10-30 22:34:55.85509309 +0000 UTC m=+3644.132786561"
Oct 30 22:34:56.022726 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:56.022641    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:56.027262 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:56.027232    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:56.738245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:56.738210    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:34:56.739397 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:34:56.739372    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:35:12.564865 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.564827    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"]
Oct 30 22:35:12.568518 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.568502    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.624661 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.624635    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-aafbca\""
Oct 30 22:35:12.660392 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.654049    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl4vh\" (UniqueName: \"kubernetes.io/projected/4bbe6772-152a-49eb-abac-9fdc770a6f30-kube-api-access-kl4vh\") pod \"payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.660392 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.654328    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bbe6772-152a-49eb-abac-9fdc770a6f30-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.675836 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.675787    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-a04bd943-dockercfg-6d77g\""
Oct 30 22:35:12.678932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.678901    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"]
Oct 30 22:35:12.755607 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.755577    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bbe6772-152a-49eb-abac-9fdc770a6f30-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.755757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.755629    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl4vh\" (UniqueName: \"kubernetes.io/projected/4bbe6772-152a-49eb-abac-9fdc770a6f30-kube-api-access-kl4vh\") pod \"payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.756277 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.756258    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bbe6772-152a-49eb-abac-9fdc770a6f30-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.764667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.764642    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"]
Oct 30 22:35:12.768231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.768215    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:12.815749 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.815699    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl4vh\" (UniqueName: \"kubernetes.io/projected/4bbe6772-152a-49eb-abac-9fdc770a6f30-kube-api-access-kl4vh\") pod \"payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.831846 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.831820    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"]
Oct 30 22:35:12.857113 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.857090    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3236de-b232-422a-85bd-fbd80761a4c7-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:12.857230 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.857216    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlwc4\" (UniqueName: \"kubernetes.io/projected/2b3236de-b232-422a-85bd-fbd80761a4c7-kube-api-access-dlwc4\") pod \"payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:12.880772 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.880746    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:12.957958 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.957919    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlwc4\" (UniqueName: \"kubernetes.io/projected/2b3236de-b232-422a-85bd-fbd80761a4c7-kube-api-access-dlwc4\") pod \"payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:12.958093 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.958034    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3236de-b232-422a-85bd-fbd80761a4c7-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:12.958785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:12.958759    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3236de-b232-422a-85bd-fbd80761a4c7-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:13.016918 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.016891    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlwc4\" (UniqueName: \"kubernetes.io/projected/2b3236de-b232-422a-85bd-fbd80761a4c7-kube-api-access-dlwc4\") pod \"payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:13.081294 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.081231    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:13.253984 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.253927    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"]
Oct 30 22:35:13.256630 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:35:13.256601    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbe6772_152a_49eb_abac_9fdc770a6f30.slice/crio-8231afc4f76f3aa3a877046a52ccb8f8bc44bfe37cab30f3e2cdedd74613132a WatchSource:0}: Error finding container 8231afc4f76f3aa3a877046a52ccb8f8bc44bfe37cab30f3e2cdedd74613132a: Status 404 returned error can't find the container with id 8231afc4f76f3aa3a877046a52ccb8f8bc44bfe37cab30f3e2cdedd74613132a
Oct 30 22:35:13.456927 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.456901    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"]
Oct 30 22:35:13.458894 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:35:13.458868    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3236de_b232_422a_85bd_fbd80761a4c7.slice/crio-27c2df09e35b9bf611798308a6292e8b1fced723d5a6cde95652478ae4cdebda WatchSource:0}: Error finding container 27c2df09e35b9bf611798308a6292e8b1fced723d5a6cde95652478ae4cdebda: Status 404 returned error can't find the container with id 27c2df09e35b9bf611798308a6292e8b1fced723d5a6cde95652478ae4cdebda
Oct 30 22:35:13.818416 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.818310    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" event={"ID":"4bbe6772-152a-49eb-abac-9fdc770a6f30","Type":"ContainerStarted","Data":"e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253"}
Oct 30 22:35:13.818416 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.818402    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:13.818886 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.818431    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" event={"ID":"4bbe6772-152a-49eb-abac-9fdc770a6f30","Type":"ContainerStarted","Data":"8231afc4f76f3aa3a877046a52ccb8f8bc44bfe37cab30f3e2cdedd74613132a"}
Oct 30 22:35:13.820054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.820028    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" event={"ID":"2b3236de-b232-422a-85bd-fbd80761a4c7","Type":"ContainerStarted","Data":"08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb"}
Oct 30 22:35:13.820189 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.820080    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" event={"ID":"2b3236de-b232-422a-85bd-fbd80761a4c7","Type":"ContainerStarted","Data":"27c2df09e35b9bf611798308a6292e8b1fced723d5a6cde95652478ae4cdebda"}
Oct 30 22:35:13.820189 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.820107    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:13.936662 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:13.936616    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" podStartSLOduration=1.936604595 podStartE2EDuration="1.936604595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:13.934860481 +0000 UTC m=+3662.212554142" watchObservedRunningTime="2026-10-30 22:35:13.936604595 +0000 UTC m=+3662.214298066"
Oct 30 22:35:14.063829 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:14.062751    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" podStartSLOduration=2.06273018 podStartE2EDuration="2.06273018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:14.062250608 +0000 UTC m=+3662.339944079" watchObservedRunningTime="2026-10-30 22:35:14.06273018 +0000 UTC m=+3662.340423650"
Oct 30 22:35:15.008902 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.008870    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"]
Oct 30 22:35:15.012740 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.012716    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.066358 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.066332    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-aafbca-serving-cert\""
Oct 30 22:35:15.088410 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.088378    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.088549 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.088416    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-metrics-certs\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.088549 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.088439    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrmfj\" (UniqueName: \"kubernetes.io/projected/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-kube-api-access-nrmfj\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.088676 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.088648    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.118679 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.118656    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"]
Oct 30 22:35:15.189821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.189783    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.189958 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.189900    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.189958 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.189927    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-metrics-certs\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.190055 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.189962    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrmfj\" (UniqueName: \"kubernetes.io/projected/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-kube-api-access-nrmfj\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.190055 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:35:15.190044    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-serving-cert: secret "maas-api-e2e-worker-single-main-aafbca-serving-cert" not found
Oct 30 22:35:15.190133 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:35:15.190111    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls podName:aef7086b-b88e-4fe9-a1b4-80d02fac0a11 nodeName:}" failed. No retries permitted until 2026-10-30 22:35:15.690094364 +0000 UTC m=+3663.967787814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls") pod "maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct" (UID: "aef7086b-b88e-4fe9-a1b4-80d02fac0a11") : secret "maas-api-e2e-worker-single-main-aafbca-serving-cert" not found
Oct 30 22:35:15.190581 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.190562    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.192192 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.192173    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-metrics-certs\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.261929 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.261843    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrmfj\" (UniqueName: \"kubernetes.io/projected/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-kube-api-access-nrmfj\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.695131 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.695097    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.697626 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.697603    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:15.925779 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:15.925749    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:16.300403 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:16.299591    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"]
Oct 30 22:35:16.835551 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:16.835513    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct" event={"ID":"aef7086b-b88e-4fe9-a1b4-80d02fac0a11","Type":"ContainerStarted","Data":"41e3540aa7b3cd0d58f7cabc0dbe4795bd93f6bf1dff554b9c01343088531aaf"}
Oct 30 22:35:17.844312 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:17.844280    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct" event={"ID":"aef7086b-b88e-4fe9-a1b4-80d02fac0a11","Type":"ContainerStarted","Data":"a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060"}
Oct 30 22:35:17.844762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:17.844339    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:17.959593 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:17.959538    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct" podStartSLOduration=3.092364245 podStartE2EDuration="3.959502378s" totalImagesPullingTime="867.138133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:17.959422635 +0000 UTC m=+3666.237116098" watchObservedRunningTime="2026-10-30 22:35:17.959502378 +0000 UTC m=+3666.237195859"
Oct 30 22:35:22.981050 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:35:22.981007    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:35:23.435217 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:23.435184    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:35:23.873322 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:23.873293    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:35:24.828576 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:24.828542    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:35:24.829075 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:24.828624    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:35:32.915325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:32.915291    5561 generic.go:356] "Generic (PLEG): container finished" podID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerID="43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de" exitCode=0
Oct 30 22:35:32.915748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:32.915362    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" event={"ID":"8c4712d6-9a59-4e7f-a07e-c97192c73465","Type":"ContainerDied","Data":"43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de"}
Oct 30 22:35:32.915748 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:32.915640    5561 scope.go:122] "RemoveContainer" containerID="43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de"
Oct 30 22:35:32.916658 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:32.916628    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:35:32.916724 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:32.916702    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:35:32.916762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:32.916736    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:35:33.921990 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:33.921954    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" event={"ID":"8c4712d6-9a59-4e7f-a07e-c97192c73465","Type":"ContainerStarted","Data":"0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49"}
Oct 30 22:35:34.022516 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:34.022484    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:35:34.022686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:34.022638    5561 patch_prober.go:29] interesting pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:35:34.022737 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:34.022681    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:35:35.022557 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:35.022527    5561 patch_prober.go:29] interesting pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:35:35.022929 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:35.022577    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:35:36.023161 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:36.023126    5561 patch_prober.go:29] interesting pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:35:36.023594 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:36.023176    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:35:37.022778 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:37.022742    5561 patch_prober.go:29] interesting pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:35:37.022957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:37.022793    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:35:38.022737 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:38.022707    5561 patch_prober.go:29] interesting pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:35:38.023206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:38.023176    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:35:39.026170 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:39.026140    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:35:39.026564 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:39.026381    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:35:39.027101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:39.027081    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:35:43.464798 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:35:43.464760    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:35:52.980606 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:35:52.980527    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:36:10.146894 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:10.146856    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:36:22.982289 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:36:22.982245    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:36:35.927470 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:35.927435    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:36:44.895054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:44.895021    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"]
Oct 30 22:36:45.008070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:45.008016    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"]
Oct 30 22:36:49.896070 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:49.895998    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" podUID="2b3236de-b232-422a-85bd-fbd80761a4c7" containerName="payload-processing" containerID="cri-o://08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb" gracePeriod=25
Oct 30 22:36:50.009386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:50.009326    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" podUID="4bbe6772-152a-49eb-abac-9fdc770a6f30" containerName="payload-pre-processing" containerID="cri-o://e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253" gracePeriod=25
Oct 30 22:36:51.753370 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:51.753325    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"]
Oct 30 22:36:51.753846 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:51.753657    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct" podUID="aef7086b-b88e-4fe9-a1b4-80d02fac0a11" containerName="maas-api" containerID="cri-o://a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060" gracePeriod=30
Oct 30 22:36:52.986164 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:36:52.986117    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:36:52.999706 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:52.999685    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:36:53.094950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.094892    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-openshift-service-ca\") pod \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") "
Oct 30 22:36:53.094950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.094928    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-metrics-certs\") pod \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") "
Oct 30 22:36:53.095101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.095039    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-kube-api-access-nrmfj\" (UniqueName: \"kubernetes.io/projected/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-kube-api-access-nrmfj\") pod \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") "
Oct 30 22:36:53.095101 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.095093    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls\") pod \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\" (UID: \"aef7086b-b88e-4fe9-a1b4-80d02fac0a11\") "
Oct 30 22:36:53.095271 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.095245    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-openshift-service-ca" pod "aef7086b-b88e-4fe9-a1b4-80d02fac0a11" (UID: "aef7086b-b88e-4fe9-a1b4-80d02fac0a11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:36:53.095484 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.095466    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:53.097032 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.097006    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-metrics-certs" pod "aef7086b-b88e-4fe9-a1b4-80d02fac0a11" (UID: "aef7086b-b88e-4fe9-a1b4-80d02fac0a11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:36:53.097120 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.097009    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-kube-api-access-nrmfj" pod "aef7086b-b88e-4fe9-a1b4-80d02fac0a11" (UID: "aef7086b-b88e-4fe9-a1b4-80d02fac0a11"). InnerVolumeSpecName "kube-api-access-nrmfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:53.097120 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.097055    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls" pod "aef7086b-b88e-4fe9-a1b4-80d02fac0a11" (UID: "aef7086b-b88e-4fe9-a1b4-80d02fac0a11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:36:53.196805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.196774    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:53.196805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.196802    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrmfj\" (UniqueName: \"kubernetes.io/projected/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-kube-api-access-nrmfj\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:53.196805 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.196813    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef7086b-b88e-4fe9-a1b4-80d02fac0a11-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:53.324627 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.324595    5561 generic.go:356] "Generic (PLEG): container finished" podID="aef7086b-b88e-4fe9-a1b4-80d02fac0a11" containerID="a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060" exitCode=0
Oct 30 22:36:53.324818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.324636    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct" event={"ID":"aef7086b-b88e-4fe9-a1b4-80d02fac0a11","Type":"ContainerDied","Data":"a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060"}
Oct 30 22:36:53.324818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.324642    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"
Oct 30 22:36:53.324818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.324691    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct" event={"ID":"aef7086b-b88e-4fe9-a1b4-80d02fac0a11","Type":"ContainerDied","Data":"41e3540aa7b3cd0d58f7cabc0dbe4795bd93f6bf1dff554b9c01343088531aaf"}
Oct 30 22:36:53.324818 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.324714    5561 scope.go:122] "RemoveContainer" containerID="a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060"
Oct 30 22:36:53.334675 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.334655    5561 scope.go:122] "RemoveContainer" containerID="a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060"
Oct 30 22:36:53.334957 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:36:53.334936    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060\": container with ID starting with a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060 not found: ID does not exist" containerID="a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060"
Oct 30 22:36:53.335033 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.334969    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060"} err="failed to get container status \"a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060\": rpc error: code = NotFound desc = could not find container \"a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060\": container with ID starting with a1925ccfef83a832c01e4553865f829b02fa27c822c570f1635d941ff9234060 not found: ID does not exist"
Oct 30 22:36:53.503469 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.503443    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"]
Oct 30 22:36:53.507385 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.507343    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-aafbca-65d8c4b7dd-x6dct"]
Oct 30 22:36:53.844149 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:53.844061    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aef7086b-b88e-4fe9-a1b4-80d02fac0a11" path="/var/lib/kubelet/pods/aef7086b-b88e-4fe9-a1b4-80d02fac0a11/volumes"
Oct 30 22:36:54.827147 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:54.827104    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" podUID="2b3236de-b232-422a-85bd-fbd80761a4c7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:36:54.827562 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:54.827114    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" podUID="4bbe6772-152a-49eb-abac-9fdc770a6f30" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:36:56.441979 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:36:56.441946    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:37:00.786006 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:00.785973    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"]
Oct 30 22:37:00.786461 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:00.786190    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy" containerID="cri-o://0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49" gracePeriod=30
Oct 30 22:37:04.826663 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:04.826625    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" podUID="2b3236de-b232-422a-85bd-fbd80761a4c7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:37:04.827049 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:04.826673    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" podUID="4bbe6772-152a-49eb-abac-9fdc770a6f30" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:37:06.030362 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.030325    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:37:06.119185 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119144    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-data\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119215    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-certs\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119280    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-envoy\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119305    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-credential-socket\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119373 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119367    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-podinfo\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119395    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-socket\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119401    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-data" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:06.119613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119432    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c4712d6-9a59-4e7f-a07e-c97192c73465-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4712d6-9a59-4e7f-a07e-c97192c73465-istiod-ca-cert\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119478    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-kube-api-access-fvljl\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-kube-api-access-fvljl\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119499    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-certs" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:06.119613 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119509    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-token\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-token\") pod \"8c4712d6-9a59-4e7f-a07e-c97192c73465\" (UID: \"8c4712d6-9a59-4e7f-a07e-c97192c73465\") "
Oct 30 22:37:06.119950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119636    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-credential-socket" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:06.119950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119787    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-socket" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:06.119950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119903    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.119950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119915    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c4712d6-9a59-4e7f-a07e-c97192c73465-istiod-ca-cert" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:06.119950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119925    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.119950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119942    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.120205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.119956    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.121530 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.121510    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-envoy" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:06.121747 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.121732    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-token" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:06.121873 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.121859    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-podinfo" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:37:06.121929 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.121889    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-kube-api-access-fvljl" pod "8c4712d6-9a59-4e7f-a07e-c97192c73465" (UID: "8c4712d6-9a59-4e7f-a07e-c97192c73465"). InnerVolumeSpecName "kube-api-access-fvljl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:06.221119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.221090    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.221119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.221115    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.221119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.221125    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4712d6-9a59-4e7f-a07e-c97192c73465-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.221332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.221135    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvljl\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-kube-api-access-fvljl\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.221332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.221144    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4712d6-9a59-4e7f-a07e-c97192c73465-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.387140 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.387068    5561 generic.go:356] "Generic (PLEG): container finished" podID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerID="0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49" exitCode=0
Oct 30 22:37:06.387140 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.387117    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"
Oct 30 22:37:06.387389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.387119    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" event={"ID":"8c4712d6-9a59-4e7f-a07e-c97192c73465","Type":"ContainerDied","Data":"0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49"}
Oct 30 22:37:06.387389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.387254    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh" event={"ID":"8c4712d6-9a59-4e7f-a07e-c97192c73465","Type":"ContainerDied","Data":"f0ca5b47a9f4589292695ac92aa2416620c5d98e40343ec026e6ada29f08e5f2"}
Oct 30 22:37:06.387389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.387294    5561 scope.go:122] "RemoveContainer" containerID="0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49"
Oct 30 22:37:06.396457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.396441    5561 scope.go:122] "RemoveContainer" containerID="43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de"
Oct 30 22:37:06.405260 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.405240    5561 scope.go:122] "RemoveContainer" containerID="0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49"
Oct 30 22:37:06.405539 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:37:06.405522    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49\": container with ID starting with 0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49 not found: ID does not exist" containerID="0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49"
Oct 30 22:37:06.405604 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.405547    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49"} err="failed to get container status \"0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49\": rpc error: code = NotFound desc = could not find container \"0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49\": container with ID starting with 0e21d749e17cfaffbf0068b52ae3b9395f6238409bf15bc3a38ef45d92696a49 not found: ID does not exist"
Oct 30 22:37:06.405604 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.405564    5561 scope.go:122] "RemoveContainer" containerID="43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de"
Oct 30 22:37:06.405781 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:37:06.405764    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de\": container with ID starting with 43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de not found: ID does not exist" containerID="43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de"
Oct 30 22:37:06.405821 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.405788    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de"} err="failed to get container status \"43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de\": rpc error: code = NotFound desc = could not find container \"43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de\": container with ID starting with 43dbb9f099743aa14c2739d4cdf85fd7b8f8e083520d3c7c6ab99f76077ae3de not found: ID does not exist"
Oct 30 22:37:06.554611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.554584    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"]
Oct 30 22:37:06.558934 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:06.558914    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh"]
Oct 30 22:37:07.853002 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:07.852961    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" path="/var/lib/kubelet/pods/8c4712d6-9a59-4e7f-a07e-c97192c73465/volumes"
Oct 30 22:37:10.148965 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.148939    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:37:10.245849 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.245823    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:37:10.264924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.264894    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b3236de-b232-422a-85bd-fbd80761a4c7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3236de-b232-422a-85bd-fbd80761a4c7-plugins-config-volume\") pod \"2b3236de-b232-422a-85bd-fbd80761a4c7\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") "
Oct 30 22:37:10.265020 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.264976    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b3236de-b232-422a-85bd-fbd80761a4c7-kube-api-access-dlwc4\" (UniqueName: \"kubernetes.io/projected/2b3236de-b232-422a-85bd-fbd80761a4c7-kube-api-access-dlwc4\") pod \"2b3236de-b232-422a-85bd-fbd80761a4c7\" (UID: \"2b3236de-b232-422a-85bd-fbd80761a4c7\") "
Oct 30 22:37:10.265270 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.265239    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b3236de-b232-422a-85bd-fbd80761a4c7-plugins-config-volume" pod "2b3236de-b232-422a-85bd-fbd80761a4c7" (UID: "2b3236de-b232-422a-85bd-fbd80761a4c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:10.265412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.265386    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3236de-b232-422a-85bd-fbd80761a4c7-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:10.267146 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.267121    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3236de-b232-422a-85bd-fbd80761a4c7-kube-api-access-dlwc4" pod "2b3236de-b232-422a-85bd-fbd80761a4c7" (UID: "2b3236de-b232-422a-85bd-fbd80761a4c7"). InnerVolumeSpecName "kube-api-access-dlwc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:10.366042 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.366014    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bbe6772-152a-49eb-abac-9fdc770a6f30-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bbe6772-152a-49eb-abac-9fdc770a6f30-plugins-config-volume\") pod \"4bbe6772-152a-49eb-abac-9fdc770a6f30\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") "
Oct 30 22:37:10.366180 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.366048    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bbe6772-152a-49eb-abac-9fdc770a6f30-kube-api-access-kl4vh\" (UniqueName: \"kubernetes.io/projected/4bbe6772-152a-49eb-abac-9fdc770a6f30-kube-api-access-kl4vh\") pod \"4bbe6772-152a-49eb-abac-9fdc770a6f30\" (UID: \"4bbe6772-152a-49eb-abac-9fdc770a6f30\") "
Oct 30 22:37:10.366460 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.366441    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlwc4\" (UniqueName: \"kubernetes.io/projected/2b3236de-b232-422a-85bd-fbd80761a4c7-kube-api-access-dlwc4\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:10.366529 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.366440    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bbe6772-152a-49eb-abac-9fdc770a6f30-plugins-config-volume" pod "4bbe6772-152a-49eb-abac-9fdc770a6f30" (UID: "4bbe6772-152a-49eb-abac-9fdc770a6f30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:10.367993 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.367972    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bbe6772-152a-49eb-abac-9fdc770a6f30-kube-api-access-kl4vh" pod "4bbe6772-152a-49eb-abac-9fdc770a6f30" (UID: "4bbe6772-152a-49eb-abac-9fdc770a6f30"). InnerVolumeSpecName "kube-api-access-kl4vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:10.409309 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.409283    5561 generic.go:356] "Generic (PLEG): container finished" podID="4bbe6772-152a-49eb-abac-9fdc770a6f30" containerID="e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253" exitCode=0
Oct 30 22:37:10.409434 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.409324    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"
Oct 30 22:37:10.409434 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.409377    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" event={"ID":"4bbe6772-152a-49eb-abac-9fdc770a6f30","Type":"ContainerDied","Data":"e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253"}
Oct 30 22:37:10.409434 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.409431    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z" event={"ID":"4bbe6772-152a-49eb-abac-9fdc770a6f30","Type":"ContainerDied","Data":"8231afc4f76f3aa3a877046a52ccb8f8bc44bfe37cab30f3e2cdedd74613132a"}
Oct 30 22:37:10.409582 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.409445    5561 scope.go:122] "RemoveContainer" containerID="e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253"
Oct 30 22:37:10.410687 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.410662    5561 generic.go:356] "Generic (PLEG): container finished" podID="2b3236de-b232-422a-85bd-fbd80761a4c7" containerID="08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb" exitCode=0
Oct 30 22:37:10.410782 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.410707    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"
Oct 30 22:37:10.410843 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.410712    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" event={"ID":"2b3236de-b232-422a-85bd-fbd80761a4c7","Type":"ContainerDied","Data":"08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb"}
Oct 30 22:37:10.410843 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.410830    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr" event={"ID":"2b3236de-b232-422a-85bd-fbd80761a4c7","Type":"ContainerDied","Data":"27c2df09e35b9bf611798308a6292e8b1fced723d5a6cde95652478ae4cdebda"}
Oct 30 22:37:10.418969 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.418951    5561 scope.go:122] "RemoveContainer" containerID="e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253"
Oct 30 22:37:10.419232 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:37:10.419217    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253\": container with ID starting with e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253 not found: ID does not exist" containerID="e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253"
Oct 30 22:37:10.419283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.419240    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253"} err="failed to get container status \"e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253\": rpc error: code = NotFound desc = could not find container \"e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253\": container with ID starting with e9717ac90f93847245f7c8b75aae857a0c3d6a286c0dc3fd98a837c3fba4d253 not found: ID does not exist"
Oct 30 22:37:10.419283 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.419256    5561 scope.go:122] "RemoveContainer" containerID="08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb"
Oct 30 22:37:10.428168 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.428148    5561 scope.go:122] "RemoveContainer" containerID="08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb"
Oct 30 22:37:10.428448 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:37:10.428427    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb\": container with ID starting with 08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb not found: ID does not exist" containerID="08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb"
Oct 30 22:37:10.428520 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.428453    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb"} err="failed to get container status \"08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb\": rpc error: code = NotFound desc = could not find container \"08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb\": container with ID starting with 08e66f83c0de028b3a6d225e9044f94d3315007289bf035f0808c9872b1271fb not found: ID does not exist"
Oct 30 22:37:10.467652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.467633    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bbe6772-152a-49eb-abac-9fdc770a6f30-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:10.467741 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.467654    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl4vh\" (UniqueName: \"kubernetes.io/projected/4bbe6772-152a-49eb-abac-9fdc770a6f30-kube-api-access-kl4vh\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:10.587780 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.587753    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"]
Oct 30 22:37:10.590079 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.590047    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z"]
Oct 30 22:37:10.700764 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.700706    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"]
Oct 30 22:37:10.704715 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:10.704694    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr"]
Oct 30 22:37:11.844977 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:11.844949    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b3236de-b232-422a-85bd-fbd80761a4c7" path="/var/lib/kubelet/pods/2b3236de-b232-422a-85bd-fbd80761a4c7/volumes"
Oct 30 22:37:11.845345 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:11.845294    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bbe6772-152a-49eb-abac-9fdc770a6f30" path="/var/lib/kubelet/pods/4bbe6772-152a-49eb-abac-9fdc770a6f30/volumes"
Oct 30 22:37:12.366451 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:12.366415    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:37:22.979978 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:37:22.979902    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:37:50.277142 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277107    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"]
Oct 30 22:37:50.277588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277514    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy"
Oct 30 22:37:50.277588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277527    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy"
Oct 30 22:37:50.277588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277560    5561 cpu_manager.go:414] "containerMap: removing container" podUID="2b3236de-b232-422a-85bd-fbd80761a4c7" containerName="payload-processing"
Oct 30 22:37:50.277588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277566    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b3236de-b232-422a-85bd-fbd80761a4c7" containerName="payload-processing"
Oct 30 22:37:50.277588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277585    5561 cpu_manager.go:414] "containerMap: removing container" podUID="4bbe6772-152a-49eb-abac-9fdc770a6f30" containerName="payload-pre-processing"
Oct 30 22:37:50.277588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277590    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbe6772-152a-49eb-abac-9fdc770a6f30" containerName="payload-pre-processing"
Oct 30 22:37:50.277784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277597    5561 cpu_manager.go:414] "containerMap: removing container" podUID="aef7086b-b88e-4fe9-a1b4-80d02fac0a11" containerName="maas-api"
Oct 30 22:37:50.277784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277602    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aef7086b-b88e-4fe9-a1b4-80d02fac0a11" containerName="maas-api"
Oct 30 22:37:50.277784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277663    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b3236de-b232-422a-85bd-fbd80761a4c7" containerName="payload-processing"
Oct 30 22:37:50.277784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277673    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy"
Oct 30 22:37:50.277784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277681    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbe6772-152a-49eb-abac-9fdc770a6f30" containerName="payload-pre-processing"
Oct 30 22:37:50.277784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277688    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="aef7086b-b88e-4fe9-a1b4-80d02fac0a11" containerName="maas-api"
Oct 30 22:37:50.277784 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.277694    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy"
Oct 30 22:37:50.280661 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.280645    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.380653 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.380615    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"]
Oct 30 22:37:50.386087 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.386065    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-12b4dd65-kserve-self-signed-certs\""
Oct 30 22:37:50.427686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.427135    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-model-cache\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.427686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.427291    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tls-certs\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.427686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.427328    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-dshm\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.427686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.427520    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-home\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.427686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.427580    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tmp-dir\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.427686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.427622    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r8rv\" (UniqueName: \"kubernetes.io/projected/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-kube-api-access-7r8rv\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529038 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.528964    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-model-cache\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529038 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529026    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tls-certs\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529046    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-dshm\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529085    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-home\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529131    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tmp-dir\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529231 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529159    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r8rv\" (UniqueName: \"kubernetes.io/projected/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-kube-api-access-7r8rv\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529476 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529433    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-model-cache\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529538 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529509    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-home\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.529603 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.529572    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tmp-dir\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.531293 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.531273    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-dshm\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.531569 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.531550    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tls-certs\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.587315 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.587284    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r8rv\" (UniqueName: \"kubernetes.io/projected/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-kube-api-access-7r8rv\") pod \"e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") " pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.593523 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.593493    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:50.984374 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:50.984331    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"]
Oct 30 22:37:50.986105 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:37:50.986079    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod944d14f6_8d53_46ef_9163_c4b0ab7b4e90.slice/crio-891f84c1a213afda82a045db5667b82b96d8239f3ebaca89ab58f8b0b99fa91e WatchSource:0}: Error finding container 891f84c1a213afda82a045db5667b82b96d8239f3ebaca89ab58f8b0b99fa91e: Status 404 returned error can't find the container with id 891f84c1a213afda82a045db5667b82b96d8239f3ebaca89ab58f8b0b99fa91e
Oct 30 22:37:51.597216 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:51.597180    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk" event={"ID":"944d14f6-8d53-46ef-9163-c4b0ab7b4e90","Type":"ContainerStarted","Data":"0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884"}
Oct 30 22:37:51.597591 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:51.597229    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk" event={"ID":"944d14f6-8d53-46ef-9163-c4b0ab7b4e90","Type":"ContainerStarted","Data":"891f84c1a213afda82a045db5667b82b96d8239f3ebaca89ab58f8b0b99fa91e"}
Oct 30 22:37:51.718999 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:51.718951    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk" podStartSLOduration=1.718940117 podStartE2EDuration="1.718940117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:51.717179856 +0000 UTC m=+3819.994873326" watchObservedRunningTime="2026-10-30 22:37:51.718940117 +0000 UTC m=+3819.996633588"
Oct 30 22:37:52.594063 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:52.594027    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:37:52.980498 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:37:52.980465    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:37:55.168390 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:55.168340    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:37:58.262504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:58.262433    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:37:58.263153 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:58.263096    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls" podUID="f212bdad-d633-4c1a-a4e5-182b295746c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:37:58.335483 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:58.335457    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:37:58.335597 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:37:58.335495    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:38:00.594580 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:38:00.594545    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:38:00.606957 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:38:00.606934    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:38:00.660365 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:38:00.660323    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:38:21.172220 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:38:21.172185    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:38:23.035179 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:38:23.035143    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:38:46.747850 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:38:46.747818    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:38:52.981817 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:38:52.981732    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:38:57.063940 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:38:57.063907    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"]
Oct 30 22:39:01.456480 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:01.456443    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:39:11.176043 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:11.176009    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:39:12.083656 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.083605    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk" podUID="944d14f6-8d53-46ef-9163-c4b0ab7b4e90" containerName="main" containerID="cri-o://0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884" gracePeriod=45
Oct 30 22:39:12.348018 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.347949    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:39:12.423239 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.422233    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tls-certs\" (UniqueName: \"kubernetes.io/secret/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tls-certs\") pod \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") "
Oct 30 22:39:12.423239 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.422336    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-home\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-home\") pod \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") "
Oct 30 22:39:12.423239 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.422416    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-model-cache\") pod \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") "
Oct 30 22:39:12.423239 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.422449    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tmp-dir\") pod \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") "
Oct 30 22:39:12.423239 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.422489    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-dshm\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-dshm\") pod \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") "
Oct 30 22:39:12.423239 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.422537    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-kube-api-access-7r8rv\" (UniqueName: \"kubernetes.io/projected/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-kube-api-access-7r8rv\") pod \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\" (UID: \"944d14f6-8d53-46ef-9163-c4b0ab7b4e90\") "
Oct 30 22:39:12.424825 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.424524    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-home" pod "944d14f6-8d53-46ef-9163-c4b0ab7b4e90" (UID: "944d14f6-8d53-46ef-9163-c4b0ab7b4e90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:12.424825 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.424593    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tmp-dir" pod "944d14f6-8d53-46ef-9163-c4b0ab7b4e90" (UID: "944d14f6-8d53-46ef-9163-c4b0ab7b4e90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:12.425115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.425065    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-model-cache" pod "944d14f6-8d53-46ef-9163-c4b0ab7b4e90" (UID: "944d14f6-8d53-46ef-9163-c4b0ab7b4e90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:12.440232 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.440187    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-dshm" pod "944d14f6-8d53-46ef-9163-c4b0ab7b4e90" (UID: "944d14f6-8d53-46ef-9163-c4b0ab7b4e90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:39:12.460472 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.457578    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tls-certs" pod "944d14f6-8d53-46ef-9163-c4b0ab7b4e90" (UID: "944d14f6-8d53-46ef-9163-c4b0ab7b4e90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:39:12.460472 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.457682    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-kube-api-access-7r8rv" pod "944d14f6-8d53-46ef-9163-c4b0ab7b4e90" (UID: "944d14f6-8d53-46ef-9163-c4b0ab7b4e90"). InnerVolumeSpecName "kube-api-access-7r8rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:12.524048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.523798    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r8rv\" (UniqueName: \"kubernetes.io/projected/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-kube-api-access-7r8rv\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:12.524048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.523834    5561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tls-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:12.524048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.523850    5561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-home\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:12.524048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.523865    5561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-model-cache\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:12.524048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.523881    5561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-tmp-dir\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:12.524048 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.523894    5561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/944d14f6-8d53-46ef-9163-c4b0ab7b4e90-dshm\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:12.535402 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.535277    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:39:12.624617 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:12.624592    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:39:13.089426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.088847    5561 generic.go:356] "Generic (PLEG): container finished" podID="944d14f6-8d53-46ef-9163-c4b0ab7b4e90" containerID="0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884" exitCode=0
Oct 30 22:39:13.089426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.088966    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk" event={"ID":"944d14f6-8d53-46ef-9163-c4b0ab7b4e90","Type":"ContainerDied","Data":"0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884"}
Oct 30 22:39:13.089426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.089004    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk" event={"ID":"944d14f6-8d53-46ef-9163-c4b0ab7b4e90","Type":"ContainerDied","Data":"891f84c1a213afda82a045db5667b82b96d8239f3ebaca89ab58f8b0b99fa91e"}
Oct 30 22:39:13.089426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.089027    5561 scope.go:122] "RemoveContainer" containerID="0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884"
Oct 30 22:39:13.096465 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.096442    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"
Oct 30 22:39:13.128282 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.127952    5561 scope.go:122] "RemoveContainer" containerID="0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884"
Oct 30 22:39:13.128994 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:39:13.128684    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884\": container with ID starting with 0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884 not found: ID does not exist" containerID="0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884"
Oct 30 22:39:13.128994 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.128741    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884"} err="failed to get container status \"0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884\": rpc error: code = NotFound desc = could not find container \"0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884\": container with ID starting with 0c2ac4f52a98bc4d61aa0d06ca341d17cb299be582b8af93ed4a31dddf78e884 not found: ID does not exist"
Oct 30 22:39:13.309382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.307137    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"]
Oct 30 22:39:13.364897 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.364819    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-12b4dd65-kserve-df7d5d99c-nd9fk"]
Oct 30 22:39:13.845383 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:13.844142    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="944d14f6-8d53-46ef-9163-c4b0ab7b4e90" path="/var/lib/kubelet/pods/944d14f6-8d53-46ef-9163-c4b0ab7b4e90/volumes"
Oct 30 22:39:22.979799 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:39:22.979764    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:39:24.161238 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:24.161210    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:39:35.144726 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:39:35.144688    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:39:52.984132 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:39:52.984090    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:40:04.864857 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:04.864814    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:40:05.187378 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.187319    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"]
Oct 30 22:40:05.187807 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.187788    5561 cpu_manager.go:414] "containerMap: removing container" podUID="944d14f6-8d53-46ef-9163-c4b0ab7b4e90" containerName="main"
Oct 30 22:40:05.187904 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.187810    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="944d14f6-8d53-46ef-9163-c4b0ab7b4e90" containerName="main"
Oct 30 22:40:05.187904 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.187844    5561 cpu_manager.go:414] "containerMap: removing container" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy"
Oct 30 22:40:05.187904 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.187852    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c4712d6-9a59-4e7f-a07e-c97192c73465" containerName="istio-proxy"
Oct 30 22:40:05.188066 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.187954    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="944d14f6-8d53-46ef-9163-c4b0ab7b4e90" containerName="main"
Oct 30 22:40:05.191501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.191420    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.246907 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.246867    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c99855-1158-42cc-bb04-18d818d6ec41-metrics-certs\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.247174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.247145    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c99855-1158-42cc-bb04-18d818d6ec41-openshift-service-ca\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.247316 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.247300    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c99855-1158-42cc-bb04-18d818d6ec41-maas-api-tls\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.247446 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.247429    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n6j7\" (UniqueName: \"kubernetes.io/projected/43c99855-1158-42cc-bb04-18d818d6ec41-kube-api-access-9n6j7\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.274115 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.274085    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"]
Oct 30 22:40:05.348668 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.348627    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n6j7\" (UniqueName: \"kubernetes.io/projected/43c99855-1158-42cc-bb04-18d818d6ec41-kube-api-access-9n6j7\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.348870 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.348697    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c99855-1158-42cc-bb04-18d818d6ec41-metrics-certs\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.348870 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.348724    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c99855-1158-42cc-bb04-18d818d6ec41-openshift-service-ca\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.348870 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.348809    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c99855-1158-42cc-bb04-18d818d6ec41-maas-api-tls\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.349487 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.349461    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c99855-1158-42cc-bb04-18d818d6ec41-openshift-service-ca\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.351121 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.351088    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c99855-1158-42cc-bb04-18d818d6ec41-metrics-certs\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.351236 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.351181    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c99855-1158-42cc-bb04-18d818d6ec41-maas-api-tls\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.407145 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.407114    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n6j7\" (UniqueName: \"kubernetes.io/projected/43c99855-1158-42cc-bb04-18d818d6ec41-kube-api-access-9n6j7\") pod \"maas-api-54959df7f7-cmhmz\" (UID: \"43c99855-1158-42cc-bb04-18d818d6ec41\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.518755 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.518702    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:05.891119 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.891095    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"]
Oct 30 22:40:05.893054 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:40:05.893022    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c99855_1158_42cc_bb04_18d818d6ec41.slice/crio-0801c8faa880f2b34f48d43e7e7027334c865a65a683c0df372182d2d9267689 WatchSource:0}: Error finding container 0801c8faa880f2b34f48d43e7e7027334c865a65a683c0df372182d2d9267689: Status 404 returned error can't find the container with id 0801c8faa880f2b34f48d43e7e7027334c865a65a683c0df372182d2d9267689
Oct 30 22:40:05.894318 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:05.894301    5561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:40:06.344511 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:06.344418    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz" event={"ID":"43c99855-1158-42cc-bb04-18d818d6ec41","Type":"ContainerStarted","Data":"0801c8faa880f2b34f48d43e7e7027334c865a65a683c0df372182d2d9267689"}
Oct 30 22:40:07.350413 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:07.350380    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz" event={"ID":"43c99855-1158-42cc-bb04-18d818d6ec41","Type":"ContainerStarted","Data":"9ddf52dd9f5e7cf7b4767e26ef262094b9790c0dea381edcd5aaf8489f2e4396"}
Oct 30 22:40:07.350889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:07.350479    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:07.468310 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:07.468259    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz" podStartSLOduration=1.650300703 podStartE2EDuration="2.4682465s" totalImagesPullingTime="817.945797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:07.466562051 +0000 UTC m=+3955.744255523" watchObservedRunningTime="2026-10-30 22:40:07.4682465 +0000 UTC m=+3955.745939972"
Oct 30 22:40:12.402084 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.402043    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"]
Oct 30 22:40:12.405734 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.405709    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.460466 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.460433    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5ff4cfa5-openshift-default-dockercfg-tfzvl\""
Oct 30 22:40:12.512234 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.512204    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"]
Oct 30 22:40:12.512925 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.512894    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-data\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.512943    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-envoy\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513059 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.512965    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-token\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.513144    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-credential-socket\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.513207    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-podinfo\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513342 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.513317    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-socket\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513432 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.513404    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-certs\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513482 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.513439    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1175da1-02d9-4052-9399-3fcf91d56c1f-istiod-ca-cert\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.513568 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.513545    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rnw9\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-kube-api-access-7rnw9\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615081 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615044    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-podinfo\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615094    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-socket\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615112    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-certs\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615131    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1175da1-02d9-4052-9399-3fcf91d56c1f-istiod-ca-cert\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615160    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rnw9\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-kube-api-access-7rnw9\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615197    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-data\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615265 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615238    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-envoy\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615624 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615266    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-token\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615624 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615335    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-credential-socket\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615738 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615636    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-data\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615738 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615705    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-certs\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615833 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615768    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-credential-socket\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615833 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615815    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-socket\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.615952 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.615933    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1175da1-02d9-4052-9399-3fcf91d56c1f-istiod-ca-cert\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.617565 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.617534    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-envoy\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.617783 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.617763    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-podinfo\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.676524 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.676451    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rnw9\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-kube-api-access-7rnw9\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.677199 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.677079    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-token\") pod \"e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") " pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:12.719507 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:12.719479    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:13.147702 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.147672    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"]
Oct 30 22:40:13.149882 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:40:13.149853    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1175da1_02d9_4052_9399_3fcf91d56c1f.slice/crio-c5a95926d5945c907da0ba33d7edb687fa87c7b6914008a69228a99a8d39145f WatchSource:0}: Error finding container c5a95926d5945c907da0ba33d7edb687fa87c7b6914008a69228a99a8d39145f: Status 404 returned error can't find the container with id c5a95926d5945c907da0ba33d7edb687fa87c7b6914008a69228a99a8d39145f
Oct 30 22:40:13.152120 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.152082    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:40:13.152223 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.152153    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:40:13.152223 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.152184    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:40:13.361768 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.361740    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54959df7f7-cmhmz"
Oct 30 22:40:13.382209 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.382174    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" event={"ID":"f1175da1-02d9-4052-9399-3fcf91d56c1f","Type":"ContainerStarted","Data":"61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4"}
Oct 30 22:40:13.382371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.382224    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" event={"ID":"f1175da1-02d9-4052-9399-3fcf91d56c1f","Type":"ContainerStarted","Data":"c5a95926d5945c907da0ba33d7edb687fa87c7b6914008a69228a99a8d39145f"}
Oct 30 22:40:13.605261 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.605161    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" podStartSLOduration=1.605147572 podStartE2EDuration="1.605147572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:13.604289189 +0000 UTC m=+3961.881982676" watchObservedRunningTime="2026-10-30 22:40:13.605147572 +0000 UTC m=+3961.882841043"
Oct 30 22:40:13.725674 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:13.722849    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:14.725317 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:14.725281    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:15.391305 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:15.391271    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:15.392117 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:15.392100    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:20.310458 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.310424    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"]
Oct 30 22:40:20.314151 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.314120    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.394723 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.394695    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-openshift-service-ca\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.394855 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.394763    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-metrics-certs\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.394855 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.394834    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.394937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.394898    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9pdb\" (UniqueName: \"kubernetes.io/projected/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-kube-api-access-c9pdb\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.420727 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.420700    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5ff4cfa5-serving-cert\""
Oct 30 22:40:20.422442 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.422419    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"]
Oct 30 22:40:20.495711 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.495685    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-openshift-service-ca\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.495862 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.495755    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-metrics-certs\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.495862 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.495799    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.495862 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.495836    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9pdb\" (UniqueName: \"kubernetes.io/projected/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-kube-api-access-c9pdb\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.496029 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:40:20.495925    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-serving-cert: secret "maas-api-e2e-ait-5ff4cfa5-serving-cert" not found
Oct 30 22:40:20.496310 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:40:20.496004    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls podName:1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b nodeName:}" failed. No retries permitted until 2026-10-30 22:40:20.995982205 +0000 UTC m=+3969.273675658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls") pod "maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c" (UID: "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b") : secret "maas-api-e2e-ait-5ff4cfa5-serving-cert" not found
Oct 30 22:40:20.496429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.496371    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-openshift-service-ca\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.498187 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.498166    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-metrics-certs\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:20.577685 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:20.577562    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9pdb\" (UniqueName: \"kubernetes.io/projected/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-kube-api-access-c9pdb\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:21.003228 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:21.003198    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:21.005597 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:21.005567    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls\") pod \"maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:21.227589 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:21.227555    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:21.597271 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:21.597247    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"]
Oct 30 22:40:21.599142 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:40:21.599110    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b0cbc4a_6fe4_4c4f_94f2_f4fb263d058b.slice/crio-430adcd6d095aff82cdb28dbbc1fa24bb21e3380712575ecfc0fe52588d9ea9f WatchSource:0}: Error finding container 430adcd6d095aff82cdb28dbbc1fa24bb21e3380712575ecfc0fe52588d9ea9f: Status 404 returned error can't find the container with id 430adcd6d095aff82cdb28dbbc1fa24bb21e3380712575ecfc0fe52588d9ea9f
Oct 30 22:40:22.344676 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:22.344645    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7975887845-vbk67"]
Oct 30 22:40:22.344864 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:22.344843    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67" podUID="a509d6f8-e623-45dc-a9d6-922a836a95f5" containerName="maas-api" containerID="cri-o://6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f" gracePeriod=30
Oct 30 22:40:22.422217 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:22.422182    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c" event={"ID":"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b","Type":"ContainerStarted","Data":"430adcd6d095aff82cdb28dbbc1fa24bb21e3380712575ecfc0fe52588d9ea9f"}
Oct 30 22:40:22.507746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:22.507705    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67" podUID="a509d6f8-e623-45dc-a9d6-922a836a95f5" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.31:8443/health\": dial tcp 10.133.0.31:8443: connect: connection refused"
Oct 30 22:40:22.980951 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:40:22.980914    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:40:23.434693 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.434655    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c" event={"ID":"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b","Type":"ContainerStarted","Data":"83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8"}
Oct 30 22:40:23.434856 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.434843    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:23.552321 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.552265    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c" podStartSLOduration=2.658816776 podStartE2EDuration="3.552251418s" totalImagesPullingTime="893.434642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:23.551378679 +0000 UTC m=+3971.829072149" watchObservedRunningTime="2026-10-30 22:40:23.552251418 +0000 UTC m=+3971.829944888"
Oct 30 22:40:23.604022 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.603994    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 22:40:23.733575 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.733489    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a509d6f8-e623-45dc-a9d6-922a836a95f5-kube-api-access-gzwbc\" (UniqueName: \"kubernetes.io/projected/a509d6f8-e623-45dc-a9d6-922a836a95f5-kube-api-access-gzwbc\") pod \"a509d6f8-e623-45dc-a9d6-922a836a95f5\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") "
Oct 30 22:40:23.733575 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.733551    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a509d6f8-e623-45dc-a9d6-922a836a95f5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a509d6f8-e623-45dc-a9d6-922a836a95f5-openshift-service-ca\") pod \"a509d6f8-e623-45dc-a9d6-922a836a95f5\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") "
Oct 30 22:40:23.733794 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.733585    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-metrics-certs\") pod \"a509d6f8-e623-45dc-a9d6-922a836a95f5\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") "
Oct 30 22:40:23.733794 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.733645    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-maas-api-tls\") pod \"a509d6f8-e623-45dc-a9d6-922a836a95f5\" (UID: \"a509d6f8-e623-45dc-a9d6-922a836a95f5\") "
Oct 30 22:40:23.733900 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.733865    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a509d6f8-e623-45dc-a9d6-922a836a95f5-openshift-service-ca" pod "a509d6f8-e623-45dc-a9d6-922a836a95f5" (UID: "a509d6f8-e623-45dc-a9d6-922a836a95f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:23.734108 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.734084    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a509d6f8-e623-45dc-a9d6-922a836a95f5-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:23.738485 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.738458    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-maas-api-tls" pod "a509d6f8-e623-45dc-a9d6-922a836a95f5" (UID: "a509d6f8-e623-45dc-a9d6-922a836a95f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:23.738485 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.738470    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a509d6f8-e623-45dc-a9d6-922a836a95f5-kube-api-access-gzwbc" pod "a509d6f8-e623-45dc-a9d6-922a836a95f5" (UID: "a509d6f8-e623-45dc-a9d6-922a836a95f5"). InnerVolumeSpecName "kube-api-access-gzwbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:23.738679 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.738466    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-metrics-certs" pod "a509d6f8-e623-45dc-a9d6-922a836a95f5" (UID: "a509d6f8-e623-45dc-a9d6-922a836a95f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:23.835332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.835296    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzwbc\" (UniqueName: \"kubernetes.io/projected/a509d6f8-e623-45dc-a9d6-922a836a95f5-kube-api-access-gzwbc\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:23.835332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.835331    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:23.835556 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:23.835342    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a509d6f8-e623-45dc-a9d6-922a836a95f5-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:24.440328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.440292    5561 generic.go:356] "Generic (PLEG): container finished" podID="a509d6f8-e623-45dc-a9d6-922a836a95f5" containerID="6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f" exitCode=0
Oct 30 22:40:24.440912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.440372    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67"
Oct 30 22:40:24.440912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.440379    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67" event={"ID":"a509d6f8-e623-45dc-a9d6-922a836a95f5","Type":"ContainerDied","Data":"6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f"}
Oct 30 22:40:24.440912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.440443    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7975887845-vbk67" event={"ID":"a509d6f8-e623-45dc-a9d6-922a836a95f5","Type":"ContainerDied","Data":"4ca3eb78ac30905907047e66ce5b20705aac66b641b72e5dbc48a7f4f95cd930"}
Oct 30 22:40:24.440912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.440475    5561 scope.go:122] "RemoveContainer" containerID="6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f"
Oct 30 22:40:24.451521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.451487    5561 scope.go:122] "RemoveContainer" containerID="6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f"
Oct 30 22:40:24.451858 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:40:24.451837    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f\": container with ID starting with 6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f not found: ID does not exist" containerID="6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f"
Oct 30 22:40:24.451947 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.451866    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f"} err="failed to get container status \"6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f\": rpc error: code = NotFound desc = could not find container \"6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f\": container with ID starting with 6943c9787a5bb7e4bbf4864e02856e3c6e4fe457179f95acb2ef501fdcfa015f not found: ID does not exist"
Oct 30 22:40:24.605887 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.605863    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7975887845-vbk67"]
Oct 30 22:40:24.609863 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:24.609842    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7975887845-vbk67"]
Oct 30 22:40:25.844183 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:25.844148    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a509d6f8-e623-45dc-a9d6-922a836a95f5" path="/var/lib/kubelet/pods/a509d6f8-e623-45dc-a9d6-922a836a95f5/volumes"
Oct 30 22:40:28.737775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:28.737739    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:40:29.465544 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:29.465508    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:40:38.516235 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:38.516200    5561 generic.go:356] "Generic (PLEG): container finished" podID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerID="61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4" exitCode=0
Oct 30 22:40:38.516743 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:38.516254    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" event={"ID":"f1175da1-02d9-4052-9399-3fcf91d56c1f","Type":"ContainerDied","Data":"61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4"}
Oct 30 22:40:38.516743 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:38.516642    5561 scope.go:122] "RemoveContainer" containerID="61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4"
Oct 30 22:40:38.517900 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:38.517865    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:40:38.517961 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:38.517938    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:40:38.517997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:38.517969    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:40:39.522751 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:39.522718    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" event={"ID":"f1175da1-02d9-4052-9399-3fcf91d56c1f","Type":"ContainerStarted","Data":"d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac"}
Oct 30 22:40:39.720126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:39.720090    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:39.720442 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:39.720418    5561 patch_prober.go:29] interesting pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:40:39.720519 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:39.720469    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:40:40.720040 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:40.720001    5561 patch_prober.go:29] interesting pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:40:40.720472 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:40.720066    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:40:41.720464 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:41.720424    5561 patch_prober.go:29] interesting pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:40:41.720950 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:41.720495    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:40:42.720132 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:42.720095    5561 patch_prober.go:29] interesting pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:40:42.720335 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:42.720164    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:40:43.720013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:43.719977    5561 patch_prober.go:29] interesting pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:40:43.720414 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:43.720044    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:40:44.724108 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:44.724077    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:44.724583 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:44.724383    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:44.725223 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:40:44.725205    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:40:52.981444 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:40:52.981406    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:41:18.780914 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:18.780876    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"]
Oct 30 22:41:18.781410 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:18.781076    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c" podUID="1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" containerName="maas-api" containerID="cri-o://83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8" gracePeriod=30
Oct 30 22:41:19.051491 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.051466    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:41:19.058078 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.058055    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-kube-api-access-c9pdb\" (UniqueName: \"kubernetes.io/projected/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-kube-api-access-c9pdb\") pod \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") "
Oct 30 22:41:19.058212 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.058145    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-metrics-certs\") pod \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") "
Oct 30 22:41:19.058212 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.058176    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls\") pod \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") "
Oct 30 22:41:19.058333 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.058218    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-openshift-service-ca\") pod \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\" (UID: \"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b\") "
Oct 30 22:41:19.058924 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.058896    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-openshift-service-ca" pod "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" (UID: "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:41:19.060890 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.060864    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls" pod "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" (UID: "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:19.060988 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.060970    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-kube-api-access-c9pdb" pod "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" (UID: "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b"). InnerVolumeSpecName "kube-api-access-c9pdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:19.061057 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.061031    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-metrics-certs" pod "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" (UID: "1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:19.159689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.159657    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9pdb\" (UniqueName: \"kubernetes.io/projected/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-kube-api-access-c9pdb\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:19.159689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.159688    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:19.159822 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.159702    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:19.159822 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.159711    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:19.725386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.724177    5561 generic.go:356] "Generic (PLEG): container finished" podID="1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" containerID="83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8" exitCode=0
Oct 30 22:41:19.725386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.724375    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c" event={"ID":"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b","Type":"ContainerDied","Data":"83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8"}
Oct 30 22:41:19.725386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.724429    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c" event={"ID":"1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b","Type":"ContainerDied","Data":"430adcd6d095aff82cdb28dbbc1fa24bb21e3380712575ecfc0fe52588d9ea9f"}
Oct 30 22:41:19.725386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.724459    5561 scope.go:122] "RemoveContainer" containerID="83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8"
Oct 30 22:41:19.725386 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.724658    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"
Oct 30 22:41:19.746588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.746041    5561 scope.go:122] "RemoveContainer" containerID="83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8"
Oct 30 22:41:19.746588 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:41:19.746558    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8\": container with ID starting with 83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8 not found: ID does not exist" containerID="83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8"
Oct 30 22:41:19.746757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.746597    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8"} err="failed to get container status \"83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8\": rpc error: code = NotFound desc = could not find container \"83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8\": container with ID starting with 83ca6709752118f7a9c017ca685aa79b7dd67ebed5bd9d06aa6ed46c554e1ed8 not found: ID does not exist"
Oct 30 22:41:19.895459 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.895428    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"]
Oct 30 22:41:19.898889 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:19.898864    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ff4cfa5-76b86d96f5-tg25c"]
Oct 30 22:41:21.843997 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:21.843959    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" path="/var/lib/kubelet/pods/1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b/volumes"
Oct 30 22:41:22.982137 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:41:22.982087    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:41:23.362691 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:23.362617    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"]
Oct 30 22:41:23.362848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:23.362815    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy" containerID="cri-o://d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac" gracePeriod=30
Oct 30 22:41:23.363980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:23.363959    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:41:28.608099 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.608071    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:41:28.642245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642187    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-credential-socket\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642245 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642241    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-token\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-token\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642456 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642312    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-socket\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642456 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642343    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-podinfo\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642456 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642423    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1175da1-02d9-4052-9399-3fcf91d56c1f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1175da1-02d9-4052-9399-3fcf91d56c1f-istiod-ca-cert\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642474    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-kube-api-access-7rnw9\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-kube-api-access-7rnw9\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642483    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-credential-socket" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:28.642621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642503    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-envoy\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642579    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-socket" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:28.642621 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642584    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-data\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642887 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642649    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-certs\") pod \"f1175da1-02d9-4052-9399-3fcf91d56c1f\" (UID: \"f1175da1-02d9-4052-9399-3fcf91d56c1f\") "
Oct 30 22:41:28.642887 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642863    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-data" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:28.643002 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.642917    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1175da1-02d9-4052-9399-3fcf91d56c1f-istiod-ca-cert" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:41:28.643138 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.643083    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-certs" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:28.643138 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.643118    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1175da1-02d9-4052-9399-3fcf91d56c1f-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.643138 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.643139    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.643424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.643157    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.643424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.643170    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.644587 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.644562    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-token" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:28.644689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.644595    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-kube-api-access-7rnw9" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "kube-api-access-7rnw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:28.644689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.644599    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-podinfo" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:41:28.644689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.644645    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-envoy" pod "f1175da1-02d9-4052-9399-3fcf91d56c1f" (UID: "f1175da1-02d9-4052-9399-3fcf91d56c1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:28.743835 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.743807    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rnw9\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-kube-api-access-7rnw9\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.743835 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.743831    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.744021 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.743841    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1175da1-02d9-4052-9399-3fcf91d56c1f-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.744021 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.743850    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.744021 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.743859    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1175da1-02d9-4052-9399-3fcf91d56c1f-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:28.776920 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.776437    5561 generic.go:356] "Generic (PLEG): container finished" podID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerID="d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac" exitCode=0
Oct 30 22:41:28.776920 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.776507    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"
Oct 30 22:41:28.776920 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.776537    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" event={"ID":"f1175da1-02d9-4052-9399-3fcf91d56c1f","Type":"ContainerDied","Data":"d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac"}
Oct 30 22:41:28.776920 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.776603    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv" event={"ID":"f1175da1-02d9-4052-9399-3fcf91d56c1f","Type":"ContainerDied","Data":"c5a95926d5945c907da0ba33d7edb687fa87c7b6914008a69228a99a8d39145f"}
Oct 30 22:41:28.776920 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.776636    5561 scope.go:122] "RemoveContainer" containerID="d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac"
Oct 30 22:41:28.791263 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.791237    5561 scope.go:122] "RemoveContainer" containerID="61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4"
Oct 30 22:41:28.799769 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.799750    5561 scope.go:122] "RemoveContainer" containerID="d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac"
Oct 30 22:41:28.800017 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:41:28.799996    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac\": container with ID starting with d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac not found: ID does not exist" containerID="d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac"
Oct 30 22:41:28.800095 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.800023    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac"} err="failed to get container status \"d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac\": rpc error: code = NotFound desc = could not find container \"d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac\": container with ID starting with d722a1a27f28710b89ca50453b6982622176566f7e5ded0e029c1ff6a2a754ac not found: ID does not exist"
Oct 30 22:41:28.800095 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.800043    5561 scope.go:122] "RemoveContainer" containerID="61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4"
Oct 30 22:41:28.800281 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:41:28.800264    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4\": container with ID starting with 61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4 not found: ID does not exist" containerID="61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4"
Oct 30 22:41:28.800331 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.800286    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4"} err="failed to get container status \"61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4\": rpc error: code = NotFound desc = could not find container \"61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4\": container with ID starting with 61cbbedf65b8527d682dfe0f128f135abd66add3007a16ffac40d8b4315e9be4 not found: ID does not exist"
Oct 30 22:41:28.942208 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.942146    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"]
Oct 30 22:41:28.948868 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:28.948845    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv"]
Oct 30 22:41:29.843781 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:29.843750    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" path="/var/lib/kubelet/pods/f1175da1-02d9-4052-9399-3fcf91d56c1f/volumes"
Oct 30 22:41:35.134280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134246    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"]
Oct 30 22:41:35.134657 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134623    5561 cpu_manager.go:414] "containerMap: removing container" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy"
Oct 30 22:41:35.134657 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134634    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy"
Oct 30 22:41:35.134657 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134653    5561 cpu_manager.go:414] "containerMap: removing container" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134660    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134671    5561 cpu_manager.go:414] "containerMap: removing container" podUID="1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" containerName="maas-api"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134676    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" containerName="maas-api"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134684    5561 cpu_manager.go:414] "containerMap: removing container" podUID="a509d6f8-e623-45dc-a9d6-922a836a95f5" containerName="maas-api"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134688    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a509d6f8-e623-45dc-a9d6-922a836a95f5" containerName="maas-api"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134754    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134764    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1175da1-02d9-4052-9399-3fcf91d56c1f" containerName="istio-proxy"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134770    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b0cbc4a-6fe4-4c4f-94f2-f4fb263d058b" containerName="maas-api"
Oct 30 22:41:35.134785 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.134776    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="a509d6f8-e623-45dc-a9d6-922a836a95f5" containerName="maas-api"
Oct 30 22:41:35.139230 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.139213    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.192665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.192630    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-f0b1ab-openshift-default-dockercfg-rk8dh\""
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197019    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-envoy\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197077    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-token\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197111    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjrsb\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-kube-api-access-wjrsb\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197179    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-certs\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197210    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-data\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197233    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-podinfo\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197254    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-credential-socket\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197332    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-socket\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.197746 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.197560    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istiod-ca-cert\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.246244 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.246208    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"]
Oct 30 22:41:35.304144 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304101    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istiod-ca-cert\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304203    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-envoy\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304241    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-token\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304332 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304269    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjrsb\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-kube-api-access-wjrsb\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304332    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-certs\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304383    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-data\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304414    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-podinfo\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304441    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-credential-socket\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.304667 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.304513    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-socket\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.305118 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.305033    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-socket\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.305757 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.305727    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-certs\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.306026 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.306004    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-data\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.306205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.305904    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istiod-ca-cert\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.306330 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.306140    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-credential-socket\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.316415 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.311979    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-podinfo\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.316415 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.314715    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-envoy\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.363949 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.363907    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjrsb\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-kube-api-access-wjrsb\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.364096 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.363981    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-token\") pod \"e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") " pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.451580 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.451501    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:35.874325 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.874299    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"]
Oct 30 22:41:35.876261 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:41:35.876238    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf50ace_8016_4350_99b6_cdae6f1ad0de.slice/crio-df3c78232290c5cc5957b9b30b6601f35211c11e1ebf6011927915a541c0b90a WatchSource:0}: Error finding container df3c78232290c5cc5957b9b30b6601f35211c11e1ebf6011927915a541c0b90a: Status 404 returned error can't find the container with id df3c78232290c5cc5957b9b30b6601f35211c11e1ebf6011927915a541c0b90a
Oct 30 22:41:35.878851 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.878815    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:35.878931 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.878884    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:35.878931 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:35.878913    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:41:36.812753 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:36.812714    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" event={"ID":"bcf50ace-8016-4350-99b6-cdae6f1ad0de","Type":"ContainerStarted","Data":"dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f"}
Oct 30 22:41:36.813286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:36.812769    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" event={"ID":"bcf50ace-8016-4350-99b6-cdae6f1ad0de","Type":"ContainerStarted","Data":"df3c78232290c5cc5957b9b30b6601f35211c11e1ebf6011927915a541c0b90a"}
Oct 30 22:41:36.983129 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:36.983076    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" podStartSLOduration=1.9830613110000002 podStartE2EDuration="1.983061311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:36.980945794 +0000 UTC m=+4045.258639267" watchObservedRunningTime="2026-10-30 22:41:36.983061311 +0000 UTC m=+4045.260754832"
Oct 30 22:41:37.451969 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:37.451938    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:37.456729 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:37.456703    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:37.817628 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:37.817543    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:37.818431 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:37.818406    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:41:43.452918 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.452871    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"]
Oct 30 22:41:43.456655 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.456634    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.512313 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.512286    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-f0b1ab\""
Oct 30 22:41:43.561828 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.561799    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"]
Oct 30 22:41:43.579965 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.579924    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.580129 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.580061    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9frdd\" (UniqueName: \"kubernetes.io/projected/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-kube-api-access-9frdd\") pod \"payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.604761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.604727    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"]
Oct 30 22:41:43.608719 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.608682    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:43.663205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.663172    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-f0b1ab-dockercfg-q7jqv\""
Oct 30 22:41:43.664869 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.664841    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"]
Oct 30 22:41:43.681457 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.681429    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.681588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.681512    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9frdd\" (UniqueName: \"kubernetes.io/projected/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-kube-api-access-9frdd\") pod \"payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.681588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.681543    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sgtn\" (UniqueName: \"kubernetes.io/projected/322a922a-a1a2-40ca-a184-f5158fd367dd-kube-api-access-5sgtn\") pod \"payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:43.681588 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.681571    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/322a922a-a1a2-40ca-a184-f5158fd367dd-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:43.682405 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.682375    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.740161 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.740098    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9frdd\" (UniqueName: \"kubernetes.io/projected/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-kube-api-access-9frdd\") pod \"payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.769382 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.769339    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:43.782848 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.782818    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sgtn\" (UniqueName: \"kubernetes.io/projected/322a922a-a1a2-40ca-a184-f5158fd367dd-kube-api-access-5sgtn\") pod \"payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:43.782962 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.782867    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/322a922a-a1a2-40ca-a184-f5158fd367dd-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:43.783585 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.783564    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/322a922a-a1a2-40ca-a184-f5158fd367dd-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:43.841912 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.841886    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sgtn\" (UniqueName: \"kubernetes.io/projected/322a922a-a1a2-40ca-a184-f5158fd367dd-kube-api-access-5sgtn\") pod \"payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:43.922512 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:43.922485    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:44.142416 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.142382    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"]
Oct 30 22:41:44.143485 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:41:44.143461    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c05060a_fc82_4c6c_ae9e_16b663e4c0d5.slice/crio-3fdef4bee972cdb9af4d745cd469300b4c594fbf099c8679e0ac59e4ca928a9f WatchSource:0}: Error finding container 3fdef4bee972cdb9af4d745cd469300b4c594fbf099c8679e0ac59e4ca928a9f: Status 404 returned error can't find the container with id 3fdef4bee972cdb9af4d745cd469300b4c594fbf099c8679e0ac59e4ca928a9f
Oct 30 22:41:44.293445 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.293421    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"]
Oct 30 22:41:44.295057 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:41:44.295035    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322a922a_a1a2_40ca_a184_f5158fd367dd.slice/crio-6dbdfb6205b9690c7b5b359b40f0f637f7043614e4d5b855bd8c77a836d85149 WatchSource:0}: Error finding container 6dbdfb6205b9690c7b5b359b40f0f637f7043614e4d5b855bd8c77a836d85149: Status 404 returned error can't find the container with id 6dbdfb6205b9690c7b5b359b40f0f637f7043614e4d5b855bd8c77a836d85149
Oct 30 22:41:44.852569 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.852531    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5" event={"ID":"322a922a-a1a2-40ca-a184-f5158fd367dd","Type":"ContainerStarted","Data":"147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1"}
Oct 30 22:41:44.853036 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.852594    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5" event={"ID":"322a922a-a1a2-40ca-a184-f5158fd367dd","Type":"ContainerStarted","Data":"6dbdfb6205b9690c7b5b359b40f0f637f7043614e4d5b855bd8c77a836d85149"}
Oct 30 22:41:44.853036 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.852621    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:41:44.854188 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.854160    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf" event={"ID":"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5","Type":"ContainerStarted","Data":"62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae"}
Oct 30 22:41:44.854328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.854207    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf" event={"ID":"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5","Type":"ContainerStarted","Data":"3fdef4bee972cdb9af4d745cd469300b4c594fbf099c8679e0ac59e4ca928a9f"}
Oct 30 22:41:44.854328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.854237    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:44.967959 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:44.967899    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5" podStartSLOduration=1.96788488 podStartE2EDuration="1.96788488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:44.967751869 +0000 UTC m=+4053.245445341" watchObservedRunningTime="2026-10-30 22:41:44.96788488 +0000 UTC m=+4053.245578351"
Oct 30 22:41:45.093092 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.093013    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf" podStartSLOduration=2.092991403 podStartE2EDuration="2.092991403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:45.090442083 +0000 UTC m=+4053.368135562" watchObservedRunningTime="2026-10-30 22:41:45.092991403 +0000 UTC m=+4053.370684875"
Oct 30 22:41:45.808881 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.808849    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"]
Oct 30 22:41:45.812538 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.812511    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:45.918652 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.918614    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-f0b1ab-serving-cert\""
Oct 30 22:41:45.919012 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.918995    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:45.919129 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.919046    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca09845d-b043-4af6-94b3-7ab9dc3879da-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:45.919129 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.919075    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vshf4\" (UniqueName: \"kubernetes.io/projected/ca09845d-b043-4af6-94b3-7ab9dc3879da-kube-api-access-vshf4\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:45.919258 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.919193    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-metrics-certs\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:45.920253 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:45.920231    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"]
Oct 30 22:41:46.020461 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.020429    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vshf4\" (UniqueName: \"kubernetes.io/projected/ca09845d-b043-4af6-94b3-7ab9dc3879da-kube-api-access-vshf4\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.020665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.020468    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-metrics-certs\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.020665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.020605    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.020665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.020654    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca09845d-b043-4af6-94b3-7ab9dc3879da-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.020864 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:41:46.020843    5561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-serving-cert: secret "maas-api-e2e-authcfg-http-f0b1ab-serving-cert" not found
Oct 30 22:41:46.020940 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:41:46.020927    5561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls podName:ca09845d-b043-4af6-94b3-7ab9dc3879da nodeName:}" failed. No retries permitted until 2026-10-30 22:41:46.520904934 +0000 UTC m=+4054.798598383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls") pod "maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q" (UID: "ca09845d-b043-4af6-94b3-7ab9dc3879da") : secret "maas-api-e2e-authcfg-http-f0b1ab-serving-cert" not found
Oct 30 22:41:46.021504 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.021481    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca09845d-b043-4af6-94b3-7ab9dc3879da-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.023072 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.023051    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-metrics-certs\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.079342 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.079270    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vshf4\" (UniqueName: \"kubernetes.io/projected/ca09845d-b043-4af6-94b3-7ab9dc3879da-kube-api-access-vshf4\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.526067 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.526029    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.528285 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.528255    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:46.725895 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:46.725854    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:47.102985 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:47.102959    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"]
Oct 30 22:41:47.104851 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:41:47.104825    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca09845d_b043_4af6_94b3_7ab9dc3879da.slice/crio-6a7ef44fd12a7d3a6aaf6ad049d191844269053b78ac7f88ff8c439f90754681 WatchSource:0}: Error finding container 6a7ef44fd12a7d3a6aaf6ad049d191844269053b78ac7f88ff8c439f90754681: Status 404 returned error can't find the container with id 6a7ef44fd12a7d3a6aaf6ad049d191844269053b78ac7f88ff8c439f90754681
Oct 30 22:41:47.868915 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:47.868883    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q" event={"ID":"ca09845d-b043-4af6-94b3-7ab9dc3879da","Type":"ContainerStarted","Data":"6a7ef44fd12a7d3a6aaf6ad049d191844269053b78ac7f88ff8c439f90754681"}
Oct 30 22:41:48.875518 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:48.875485    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q" event={"ID":"ca09845d-b043-4af6-94b3-7ab9dc3879da","Type":"ContainerStarted","Data":"43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc"}
Oct 30 22:41:48.875925 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:48.875607    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:48.993393 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:48.993317    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q" podStartSLOduration=3.051193634 podStartE2EDuration="3.993305985s" totalImagesPullingTime="942.112351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:48.993140626 +0000 UTC m=+4057.270834099" watchObservedRunningTime="2026-10-30 22:41:48.993305985 +0000 UTC m=+4057.270999455"
Oct 30 22:41:52.980408 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:41:52.980323    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:41:54.040643 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:54.040609    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:41:54.888176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:54.888138    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:41:55.893771 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:55.893735    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:41:55.894166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:41:55.893830    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:42:03.950754 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:03.950723    5561 generic.go:356] "Generic (PLEG): container finished" podID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerID="dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f" exitCode=0
Oct 30 22:42:03.951193 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:03.950765    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" event={"ID":"bcf50ace-8016-4350-99b6-cdae6f1ad0de","Type":"ContainerDied","Data":"dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f"}
Oct 30 22:42:03.951193 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:03.951059    5561 scope.go:122] "RemoveContainer" containerID="dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f"
Oct 30 22:42:03.952164 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:03.952135    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:42:03.952219 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:03.952205    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:42:03.952255 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:03.952236    5561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:42:04.961937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:04.961897    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" event={"ID":"bcf50ace-8016-4350-99b6-cdae6f1ad0de","Type":"ContainerStarted","Data":"9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a"}
Oct 30 22:42:05.468394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:05.457750    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:42:05.468394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:05.457902    5561 patch_prober.go:29] interesting pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:42:05.468394 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:05.457946    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:42:06.452830 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:06.452797    5561 patch_prober.go:29] interesting pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:42:06.453426 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:06.453392    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:42:07.466898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:07.466452    5561 patch_prober.go:29] interesting pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:42:07.466898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:07.466515    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:42:07.818239 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:07.818148    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:42:08.463376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:08.459661    5561 patch_prober.go:29] interesting pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:42:08.463376 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:08.459723    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:42:09.458760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:09.458715    5561 patch_prober.go:29] interesting pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:42:09.459801 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:09.459342    5561 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:42:10.456795 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:10.456765    5561 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:42:10.457770 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:42:10.457752    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:42:22.980705 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:42:22.980654    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:42:52.979602 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:42:52.979562    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:43:22.979775 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:43:22.979692    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:43:35.570620 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:35.570585    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"]
Oct 30 22:43:35.680563 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:35.680533    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"]
Oct 30 22:43:40.571343 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.571285    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5" podUID="322a922a-a1a2-40ca-a184-f5158fd367dd" containerName="payload-processing" containerID="cri-o://147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1" gracePeriod=25
Oct 30 22:43:40.681495 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.681447    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf" podUID="2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" containerName="payload-pre-processing" containerID="cri-o://62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae" gracePeriod=25
Oct 30 22:43:40.821196 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.821168    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:43:40.937534 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.937508    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/322a922a-a1a2-40ca-a184-f5158fd367dd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/322a922a-a1a2-40ca-a184-f5158fd367dd-plugins-config-volume\") pod \"322a922a-a1a2-40ca-a184-f5158fd367dd\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") "
Oct 30 22:43:40.937722 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.937707    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/322a922a-a1a2-40ca-a184-f5158fd367dd-kube-api-access-5sgtn\" (UniqueName: \"kubernetes.io/projected/322a922a-a1a2-40ca-a184-f5158fd367dd-kube-api-access-5sgtn\") pod \"322a922a-a1a2-40ca-a184-f5158fd367dd\" (UID: \"322a922a-a1a2-40ca-a184-f5158fd367dd\") "
Oct 30 22:43:40.938004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.937895    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/322a922a-a1a2-40ca-a184-f5158fd367dd-plugins-config-volume" pod "322a922a-a1a2-40ca-a184-f5158fd367dd" (UID: "322a922a-a1a2-40ca-a184-f5158fd367dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:43:40.938560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.938335    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/322a922a-a1a2-40ca-a184-f5158fd367dd-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:40.939054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.939033    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:43:40.941060 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:40.941032    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/322a922a-a1a2-40ca-a184-f5158fd367dd-kube-api-access-5sgtn" pod "322a922a-a1a2-40ca-a184-f5158fd367dd" (UID: "322a922a-a1a2-40ca-a184-f5158fd367dd"). InnerVolumeSpecName "kube-api-access-5sgtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:43:41.038911 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.038888    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-plugins-config-volume\") pod \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") "
Oct 30 22:43:41.039045 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.038956    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-kube-api-access-9frdd\" (UniqueName: \"kubernetes.io/projected/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-kube-api-access-9frdd\") pod \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\" (UID: \"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5\") "
Oct 30 22:43:41.039205 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.039192    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sgtn\" (UniqueName: \"kubernetes.io/projected/322a922a-a1a2-40ca-a184-f5158fd367dd-kube-api-access-5sgtn\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:41.039328 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.039298    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-plugins-config-volume" pod "2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" (UID: "2c05060a-fc82-4c6c-ae9e-16b663e4c0d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:43:41.041031 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.041014    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-kube-api-access-9frdd" pod "2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" (UID: "2c05060a-fc82-4c6c-ae9e-16b663e4c0d5"). InnerVolumeSpecName "kube-api-access-9frdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:43:41.139908 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.139881    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9frdd\" (UniqueName: \"kubernetes.io/projected/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-kube-api-access-9frdd\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:41.139908 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.139909    5561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5-plugins-config-volume\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:41.426692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.426610    5561 generic.go:356] "Generic (PLEG): container finished" podID="2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" containerID="62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae" exitCode=0
Oct 30 22:43:41.426692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.426645    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf" event={"ID":"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5","Type":"ContainerDied","Data":"62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae"}
Oct 30 22:43:41.426692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.426673    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"
Oct 30 22:43:41.427041 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.426696    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf" event={"ID":"2c05060a-fc82-4c6c-ae9e-16b663e4c0d5","Type":"ContainerDied","Data":"3fdef4bee972cdb9af4d745cd469300b4c594fbf099c8679e0ac59e4ca928a9f"}
Oct 30 22:43:41.427041 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.426722    5561 scope.go:122] "RemoveContainer" containerID="62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae"
Oct 30 22:43:41.428028 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.428006    5561 generic.go:356] "Generic (PLEG): container finished" podID="322a922a-a1a2-40ca-a184-f5158fd367dd" containerID="147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1" exitCode=0
Oct 30 22:43:41.428130 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.428048    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"
Oct 30 22:43:41.428130 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.428106    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5" event={"ID":"322a922a-a1a2-40ca-a184-f5158fd367dd","Type":"ContainerDied","Data":"147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1"}
Oct 30 22:43:41.428250 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.428155    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5" event={"ID":"322a922a-a1a2-40ca-a184-f5158fd367dd","Type":"ContainerDied","Data":"6dbdfb6205b9690c7b5b359b40f0f637f7043614e4d5b855bd8c77a836d85149"}
Oct 30 22:43:41.438020 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.438000    5561 scope.go:122] "RemoveContainer" containerID="62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae"
Oct 30 22:43:41.438643 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:43:41.438616    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae\": container with ID starting with 62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae not found: ID does not exist" containerID="62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae"
Oct 30 22:43:41.438735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.438653    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae"} err="failed to get container status \"62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae\": rpc error: code = NotFound desc = could not find container \"62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae\": container with ID starting with 62f435b586ccc7efb989c004131e3f8adf3bb5b1dfce027c2f6bb39b4c533aae not found: ID does not exist"
Oct 30 22:43:41.438735 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.438676    5561 scope.go:122] "RemoveContainer" containerID="147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1"
Oct 30 22:43:41.446615 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.446594    5561 scope.go:122] "RemoveContainer" containerID="147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1"
Oct 30 22:43:41.446869 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:43:41.446848    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1\": container with ID starting with 147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1 not found: ID does not exist" containerID="147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1"
Oct 30 22:43:41.446937 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.446875    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1"} err="failed to get container status \"147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1\": rpc error: code = NotFound desc = could not find container \"147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1\": container with ID starting with 147e99067d54e3861f614ba5b98ca93b0dce630256dd416d7fa1b07afa1494a1 not found: ID does not exist"
Oct 30 22:43:41.592232 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.592204    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"]
Oct 30 22:43:41.595605 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.595584    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5"]
Oct 30 22:43:41.705860 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.705793    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"]
Oct 30 22:43:41.710054 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.710030    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf"]
Oct 30 22:43:41.843854 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.843813    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" path="/var/lib/kubelet/pods/2c05060a-fc82-4c6c-ae9e-16b663e4c0d5/volumes"
Oct 30 22:43:41.844286 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:41.844266    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="322a922a-a1a2-40ca-a184-f5158fd367dd" path="/var/lib/kubelet/pods/322a922a-a1a2-40ca-a184-f5158fd367dd/volumes"
Oct 30 22:43:47.608049 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:47.608013    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"]
Oct 30 22:43:47.608491 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:47.608235    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q" podUID="ca09845d-b043-4af6-94b3-7ab9dc3879da" containerName="maas-api" containerID="cri-o://43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc" gracePeriod=30
Oct 30 22:43:47.883063 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:47.883035    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:43:48.013932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.012767    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca09845d-b043-4af6-94b3-7ab9dc3879da-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca09845d-b043-4af6-94b3-7ab9dc3879da-openshift-service-ca\") pod \"ca09845d-b043-4af6-94b3-7ab9dc3879da\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") "
Oct 30 22:43:48.013932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.012861    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-metrics-certs\") pod \"ca09845d-b043-4af6-94b3-7ab9dc3879da\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") "
Oct 30 22:43:48.013932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.012924    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca09845d-b043-4af6-94b3-7ab9dc3879da-kube-api-access-vshf4\" (UniqueName: \"kubernetes.io/projected/ca09845d-b043-4af6-94b3-7ab9dc3879da-kube-api-access-vshf4\") pod \"ca09845d-b043-4af6-94b3-7ab9dc3879da\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") "
Oct 30 22:43:48.013932 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.013158    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls\") pod \"ca09845d-b043-4af6-94b3-7ab9dc3879da\" (UID: \"ca09845d-b043-4af6-94b3-7ab9dc3879da\") "
Oct 30 22:43:48.015874 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.015843    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca09845d-b043-4af6-94b3-7ab9dc3879da-openshift-service-ca" pod "ca09845d-b043-4af6-94b3-7ab9dc3879da" (UID: "ca09845d-b043-4af6-94b3-7ab9dc3879da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:43:48.029930 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.029877    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-metrics-certs" pod "ca09845d-b043-4af6-94b3-7ab9dc3879da" (UID: "ca09845d-b043-4af6-94b3-7ab9dc3879da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:43:48.030166 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.030134    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls" pod "ca09845d-b043-4af6-94b3-7ab9dc3879da" (UID: "ca09845d-b043-4af6-94b3-7ab9dc3879da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:43:48.032521 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.032483    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca09845d-b043-4af6-94b3-7ab9dc3879da-kube-api-access-vshf4" pod "ca09845d-b043-4af6-94b3-7ab9dc3879da" (UID: "ca09845d-b043-4af6-94b3-7ab9dc3879da"). InnerVolumeSpecName "kube-api-access-vshf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:43:48.114455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.114387    5561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-maas-api-tls\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:48.114455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.114411    5561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca09845d-b043-4af6-94b3-7ab9dc3879da-openshift-service-ca\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:48.114455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.114422    5561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca09845d-b043-4af6-94b3-7ab9dc3879da-metrics-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:48.114455 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.114435    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vshf4\" (UniqueName: \"kubernetes.io/projected/ca09845d-b043-4af6-94b3-7ab9dc3879da-kube-api-access-vshf4\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:43:48.461788 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.461750    5561 generic.go:356] "Generic (PLEG): container finished" podID="ca09845d-b043-4af6-94b3-7ab9dc3879da" containerID="43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc" exitCode=0
Oct 30 22:43:48.461788 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.461789    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q" event={"ID":"ca09845d-b043-4af6-94b3-7ab9dc3879da","Type":"ContainerDied","Data":"43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc"}
Oct 30 22:43:48.462029 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.461834    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q" event={"ID":"ca09845d-b043-4af6-94b3-7ab9dc3879da","Type":"ContainerDied","Data":"6a7ef44fd12a7d3a6aaf6ad049d191844269053b78ac7f88ff8c439f90754681"}
Oct 30 22:43:48.462029 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.461863    5561 scope.go:122] "RemoveContainer" containerID="43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc"
Oct 30 22:43:48.462029 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.461791    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"
Oct 30 22:43:48.471212 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.471196    5561 scope.go:122] "RemoveContainer" containerID="43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc"
Oct 30 22:43:48.471438 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:43:48.471420    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc\": container with ID starting with 43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc not found: ID does not exist" containerID="43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc"
Oct 30 22:43:48.471501 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.471446    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc"} err="failed to get container status \"43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc\": rpc error: code = NotFound desc = could not find container \"43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc\": container with ID starting with 43db326daef67ebc278607fe87f23e5fc72dcb11ac3462b5fde24760aab061dc not found: ID does not exist"
Oct 30 22:43:48.637789 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.637756    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"]
Oct 30 22:43:48.639650 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:48.639630    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-f0b1ab-db7864fdd-zfk7q"]
Oct 30 22:43:49.844416 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:49.844376    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca09845d-b043-4af6-94b3-7ab9dc3879da" path="/var/lib/kubelet/pods/ca09845d-b043-4af6-94b3-7ab9dc3879da/volumes"
Oct 30 22:43:52.981014 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:43:52.980978    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:43:54.467510 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:54.467476    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:43:56.925167 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:56.925134    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"]
Oct 30 22:43:56.925666 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:43:56.925405    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy" containerID="cri-o://9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a" gracePeriod=30
Oct 30 22:44:02.174537 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.174506    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:44:02.261543 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261454    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-token\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-token\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.261543 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261528    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-podinfo\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.261762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261584    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-envoy\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.261762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261622    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-data\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.261762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261666    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-credential-socket\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.261762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261706    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-kube-api-access-wjrsb\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-kube-api-access-wjrsb\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.261762 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261742    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istiod-ca-cert\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.262018 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261781    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-socket\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.262018 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261838    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-certs\") pod \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\" (UID: \"bcf50ace-8016-4350-99b6-cdae6f1ad0de\") "
Oct 30 22:44:02.262018 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261980    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-credential-socket" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:02.262018 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.261992    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-data" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:02.262236 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.262199    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-certs" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:02.262389 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.262340    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.262490 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.262394    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.262490 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.262411    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.262490 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.262444    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istiod-ca-cert" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:44:02.262617 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.262507    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-socket" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:02.263856 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.263827    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-podinfo" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:44:02.263956 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.263904    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-envoy" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:02.264163 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.264143    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-kube-api-access-wjrsb" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "kube-api-access-wjrsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:44:02.264247 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.264169    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-token" pod "bcf50ace-8016-4350-99b6-cdae6f1ad0de" (UID: "bcf50ace-8016-4350-99b6-cdae6f1ad0de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:44:02.363471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.363441    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.363471 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.363469    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjrsb\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-kube-api-access-wjrsb\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.363692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.363481    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.363692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.363490    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf50ace-8016-4350-99b6-cdae6f1ad0de-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.363692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.363499    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.363692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.363508    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf50ace-8016-4350-99b6-cdae6f1ad0de-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:44:02.529449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.529365    5561 generic.go:356] "Generic (PLEG): container finished" podID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerID="9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a" exitCode=0
Oct 30 22:44:02.529449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.529393    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" event={"ID":"bcf50ace-8016-4350-99b6-cdae6f1ad0de","Type":"ContainerDied","Data":"9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a"}
Oct 30 22:44:02.529449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.529427    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"
Oct 30 22:44:02.529686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.529477    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk" event={"ID":"bcf50ace-8016-4350-99b6-cdae6f1ad0de","Type":"ContainerDied","Data":"df3c78232290c5cc5957b9b30b6601f35211c11e1ebf6011927915a541c0b90a"}
Oct 30 22:44:02.529686 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.529499    5561 scope.go:122] "RemoveContainer" containerID="9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a"
Oct 30 22:44:02.539010 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.538994    5561 scope.go:122] "RemoveContainer" containerID="dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f"
Oct 30 22:44:02.546695 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.546674    5561 scope.go:122] "RemoveContainer" containerID="9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a"
Oct 30 22:44:02.546936 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:44:02.546915    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a\": container with ID starting with 9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a not found: ID does not exist" containerID="9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a"
Oct 30 22:44:02.546990 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.546944    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a"} err="failed to get container status \"9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a\": rpc error: code = NotFound desc = could not find container \"9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a\": container with ID starting with 9a245a596eb2e73fcdc081613f00186b5c8821a3452f38bff4e00bbb50fd852a not found: ID does not exist"
Oct 30 22:44:02.546990 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.546962    5561 scope.go:122] "RemoveContainer" containerID="dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f"
Oct 30 22:44:02.547193 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:44:02.547175    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f\": container with ID starting with dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f not found: ID does not exist" containerID="dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f"
Oct 30 22:44:02.547240 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.547199    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f"} err="failed to get container status \"dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f\": rpc error: code = NotFound desc = could not find container \"dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f\": container with ID starting with dfe19d17d9c98782f6f658afa342f8201cdade4f917a3221f87039fe3afdce0f not found: ID does not exist"
Oct 30 22:44:02.695474 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.695445    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"]
Oct 30 22:44:02.698174 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:02.698156    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk"]
Oct 30 22:44:03.843980 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:03.843946    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" path="/var/lib/kubelet/pods/bcf50ace-8016-4350-99b6-cdae6f1ad0de/volumes"
Oct 30 22:44:04.758278 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:04.758248    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:44:12.602871 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:12.602840    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:44:12.655021 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:12.654993    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:44:16.123233 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:16.123195    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:44:22.979714 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:44:22.979679    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:44:37.061665 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:44:37.061632    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:44:52.981811 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:44:52.981736    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:45:09.578657 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:45:09.578623    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-t7q4m"]
Oct 30 22:45:22.984999 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:45:22.984958    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:45:31.351154 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:45:31.351119    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7567487b8b-trv6g_2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1/manager/0.log"
Oct 30 22:45:31.954557 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:45:31.954527    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-2bsq2_04432384-5c7b-4cf0-92eb-87d4beac19be/manager/1.log"
Oct 30 22:45:32.227411 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:45:32.227323    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-hrf8w_6ae5b07c-0a6e-4ddb-aa67-741879d695b1/proxy/0.log"
Oct 30 22:45:32.646402 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:45:32.646378    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-2bsq2_04432384-5c7b-4cf0-92eb-87d4beac19be/manager/0.log"
Oct 30 22:45:33.216608 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:45:33.216569    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54959df7f7-cmhmz_43c99855-1158-42cc-bb04-18d818d6ec41/maas-api/0.log"
Oct 30 22:45:47.697057 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:45:47.697025    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-t7q4m_a450e568-8366-4af9-b113-0590beb38605/limitador/0.log"
Oct 30 22:45:52.980870 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:45:52.980830    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:46:09.202806 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:09.202773    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-clbjj_37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f/discovery/0.log"
Oct 30 22:46:09.501535 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:09.501462    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c46d956fb-42ws7_b1c830f7-14d0-4673-83b5-cea2dd6a33c9/kube-auth-proxy/0.log"
Oct 30 22:46:10.519210 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:10.519185    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bmmjn_ec4d573a-ec69-4842-b77f-8c3932e1efef/istio-proxy/0.log"
Oct 30 22:46:11.078093 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:11.078057    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-nvkls_f212bdad-d633-4c1a-a4e5-182b295746c7/istio-proxy/0.log"
Oct 30 22:46:11.639935 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:11.639903    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-pxw88_ce7446d0-d8a5-4e3f-9895-49698bfa488d/istio-proxy/0.log"
Oct 30 22:46:14.161026 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:14.160999    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7_8bcf17cc-fbd2-471e-af39-384080e3d198/main/0.log"
Oct 30 22:46:14.432013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:14.431944    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-z2gfw_ee336b48-8449-45ec-a82d-7931efe627ee/main/0.log"
Oct 30 22:46:14.930008 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:14.929977    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-9xc8j_1dff099d-a1aa-41e4-b2c3-3bd96da42601/main/0.log"
Oct 30 22:46:15.145692 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:15.145665    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm_a23bab1a-e64d-4c29-9e1b-941b7fdbdb34/main/0.log"
Oct 30 22:46:15.932298 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:15.932267    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6_1301ca09-349b-414d-bdf2-8297fa20c2cd/main/0.log"
Oct 30 22:46:22.981506 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:46:22.981419    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:46:37.785437 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:37.785397    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7567487b8b-trv6g_2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1/manager/0.log"
Oct 30 22:46:38.281899 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:38.281855    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-2bsq2_04432384-5c7b-4cf0-92eb-87d4beac19be/manager/1.log"
Oct 30 22:46:38.540186 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:38.540095    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-hrf8w_6ae5b07c-0a6e-4ddb-aa67-741879d695b1/proxy/0.log"
Oct 30 22:46:39.079012 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:39.078981    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54959df7f7-cmhmz_43c99855-1158-42cc-bb04-18d818d6ec41/maas-api/0.log"
Oct 30 22:46:43.014926 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:43.014895    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-t7q4m_a450e568-8366-4af9-b113-0590beb38605/limitador/0.log"
Oct 30 22:46:44.265906 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:44.265872    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-clbjj_37ba2ae5-f6f0-46b1-8315-aa0c5a801e3f/discovery/0.log"
Oct 30 22:46:44.536834 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:44.536762    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c46d956fb-42ws7_b1c830f7-14d0-4673-83b5-cea2dd6a33c9/kube-auth-proxy/0.log"
Oct 30 22:46:45.541133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:45.541107    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bmmjn_ec4d573a-ec69-4842-b77f-8c3932e1efef/istio-proxy/0.log"
Oct 30 22:46:46.095535 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:46.095507    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-nvkls_f212bdad-d633-4c1a-a4e5-182b295746c7/istio-proxy/0.log"
Oct 30 22:46:46.652064 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:46.652027    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-pxw88_ce7446d0-d8a5-4e3f-9895-49698bfa488d/istio-proxy/0.log"
Oct 30 22:46:48.924633 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:48.924603    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-4cgt7_8bcf17cc-fbd2-471e-af39-384080e3d198/main/0.log"
Oct 30 22:46:49.153875 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:49.153847    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-z2gfw_ee336b48-8449-45ec-a82d-7931efe627ee/main/0.log"
Oct 30 22:46:49.635247 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:49.635221    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-9xc8j_1dff099d-a1aa-41e4-b2c3-3bd96da42601/main/0.log"
Oct 30 22:46:49.844554 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:49.844524    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b89njjm_a23bab1a-e64d-4c29-9e1b-941b7fdbdb34/main/0.log"
Oct 30 22:46:50.619995 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:46:50.619969    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-6xjq6_1301ca09-349b-414d-bdf2-8297fa20c2cd/main/0.log"
Oct 30 22:46:52.981298 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:46:52.981263    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:47:22.980994 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:47:22.980957    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:47:29.152636 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:47:29.152604    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7567487b8b-trv6g_2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1/manager/0.log"
Oct 30 22:47:29.671096 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:47:29.671062    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7567487b8b-trv6g_2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1/manager/0.log"
Oct 30 22:47:52.979562 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:47:52.979485    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:48:22.981346 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:48:22.981312    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:48:52.981299 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:48:52.981261    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:49:12.632415 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:49:12.632387    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:49:12.684411 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:49:12.684387    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hsgh_e0ceab3b-942e-4950-bb5e-96f62e431365/ovn-acl-logging/0.log"
Oct 30 22:49:22.980124 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:49:22.980041    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:49:52.983565 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:49:52.983523    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:50:18.249004 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:18.248970    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7567487b8b-trv6g_2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1/manager/0.log"
Oct 30 22:50:18.813761 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:18.813729    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7567487b8b-trv6g_2980b3fe-2c79-46a2-a8e9-11fe52d2b3e1/manager/0.log"
Oct 30 22:50:22.981152 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:50:22.981119    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:50:29.036019 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:29.035989    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2glxt_1a7bdebe-605a-432d-8cc7-91fc5417158d/global-pull-secret-syncer/0.log"
Oct 30 22:50:33.291600 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:33.291563    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qkddr_49415b6f-832d-4e10-b7f9-e0b773574b30/konnectivity-agent/0.log"
Oct 30 22:50:34.617852 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:34.617820    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal_9a9ea5e58d330910c40e8b399ca003fd/haproxy/0.log"
Oct 30 22:50:39.144907 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.144872    5561 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"]
Oct 30 22:50:39.145304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145271    5561 cpu_manager.go:414] "containerMap: removing container" podUID="322a922a-a1a2-40ca-a184-f5158fd367dd" containerName="payload-processing"
Oct 30 22:50:39.145304 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145288    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="322a922a-a1a2-40ca-a184-f5158fd367dd" containerName="payload-processing"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145312    5561 cpu_manager.go:414] "containerMap: removing container" podUID="ca09845d-b043-4af6-94b3-7ab9dc3879da" containerName="maas-api"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145317    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca09845d-b043-4af6-94b3-7ab9dc3879da" containerName="maas-api"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145328    5561 cpu_manager.go:414] "containerMap: removing container" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145334    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145346    5561 cpu_manager.go:414] "containerMap: removing container" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145362    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145372    5561 cpu_manager.go:414] "containerMap: removing container" podUID="2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" containerName="payload-pre-processing"
Oct 30 22:50:39.145412 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145376    5561 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" containerName="payload-pre-processing"
Oct 30 22:50:39.145689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145437    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca09845d-b043-4af6-94b3-7ab9dc3879da" containerName="maas-api"
Oct 30 22:50:39.145689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145445    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy"
Oct 30 22:50:39.145689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145451    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcf50ace-8016-4350-99b6-cdae6f1ad0de" containerName="istio-proxy"
Oct 30 22:50:39.145689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145459    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c05060a-fc82-4c6c-ae9e-16b663e4c0d5" containerName="payload-pre-processing"
Oct 30 22:50:39.145689 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.145466    5561 memory_manager.go:365] "RemoveStaleState removing state" podUID="322a922a-a1a2-40ca-a184-f5158fd367dd" containerName="payload-processing"
Oct 30 22:50:39.148300 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.148285    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.201062 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.201036    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phvzg\"/\"openshift-service-ca.crt\""
Oct 30 22:50:39.201176 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.201139    5561 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phvzg\"/\"kube-root-ca.crt\""
Oct 30 22:50:39.251679 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.251656    5561 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-phvzg\"/\"default-dockercfg-t97cc\""
Oct 30 22:50:39.253058 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.253038    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"]
Oct 30 22:50:39.305327 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.305298    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-proc\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.305489 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.305329    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-sys\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.305489 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.305474    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-podres\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.305595 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.305510    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6svx\" (UniqueName: \"kubernetes.io/projected/0ccf0cfd-88cf-4ce3-8437-576e0488d812-kube-api-access-l6svx\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.305595 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.305573    5561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-lib-modules\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406518 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406454    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-proc\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406518 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406485    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-sys\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406701 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406531    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-podres\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406701 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406547    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6svx\" (UniqueName: \"kubernetes.io/projected/0ccf0cfd-88cf-4ce3-8437-576e0488d812-kube-api-access-l6svx\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406701 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406572    5561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-lib-modules\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406701 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406578    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-proc\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406701 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406659    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-sys\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406701 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406683    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-lib-modules\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.406922 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.406720    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0ccf0cfd-88cf-4ce3-8437-576e0488d812-podres\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.464126 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.464098    5561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6svx\" (UniqueName: \"kubernetes.io/projected/0ccf0cfd-88cf-4ce3-8437-576e0488d812-kube-api-access-l6svx\") pod \"perf-node-gather-daemonset-hcxmd\" (UID: \"0ccf0cfd-88cf-4ce3-8437-576e0488d812\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:39.760421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:39.760339    5561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:40.130993 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:40.130962    5561 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"]
Oct 30 22:50:40.132475 ip-10-0-135-39 kubenswrapper[5561]: W1030 22:50:40.132448    5561 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0ccf0cfd_88cf_4ce3_8437_576e0488d812.slice/crio-c17eff410a0e540d2ca4f4040af7a1486e32965996ec2d9fac4c27316fa7ab51 WatchSource:0}: Error finding container c17eff410a0e540d2ca4f4040af7a1486e32965996ec2d9fac4c27316fa7ab51: Status 404 returned error can't find the container with id c17eff410a0e540d2ca4f4040af7a1486e32965996ec2d9fac4c27316fa7ab51
Oct 30 22:50:40.373132 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:40.373094    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd" event={"ID":"0ccf0cfd-88cf-4ce3-8437-576e0488d812","Type":"ContainerStarted","Data":"80e211dfe40abb127464e3ee40114811694299dd2416c8b4fa4eef396e1da68d"}
Oct 30 22:50:40.373573 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:40.373153    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd" event={"ID":"0ccf0cfd-88cf-4ce3-8437-576e0488d812","Type":"ContainerStarted","Data":"c17eff410a0e540d2ca4f4040af7a1486e32965996ec2d9fac4c27316fa7ab51"}
Oct 30 22:50:40.373573 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:40.373178    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:40.487687 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:40.487608    5561 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd" podStartSLOduration=1.4875948 podStartE2EDuration="1.4875948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:50:40.486458103 +0000 UTC m=+4588.764151575" watchObservedRunningTime="2026-10-30 22:50:40.4875948 +0000 UTC m=+4588.765288335"
Oct 30 22:50:40.489743 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:40.489723    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-t7q4m_a450e568-8366-4af9-b113-0590beb38605/limitador/0.log"
Oct 30 22:50:41.398708 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:41.398677    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"]
Oct 30 22:50:41.399133 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:41.398853    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" containerID="cri-o://1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303" gracePeriod=30
Oct 30 22:50:41.449206 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:41.449175    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"]
Oct 30 22:50:41.449375 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:41.449344    5561 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" containerID="cri-o://dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923" gracePeriod=30
Oct 30 22:50:41.621878 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:41.621843    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:50:41.622039 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:41.621895    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:50:42.295359 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:42.295322    5561 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:50:42.295560 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:42.295392    5561 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:50:43.495977 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:43.495944    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96/alertmanager/0.log"
Oct 30 22:50:43.914371 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:43.914327    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96/config-reloader/0.log"
Oct 30 22:50:44.325961 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:44.325880    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96/kube-rbac-proxy-web/0.log"
Oct 30 22:50:44.736456 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:44.736420    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96/kube-rbac-proxy/0.log"
Oct 30 22:50:45.151240 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:45.151208    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96/kube-rbac-proxy-metric/0.log"
Oct 30 22:50:45.562744 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:45.562669    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96/prom-label-proxy/0.log"
Oct 30 22:50:45.978105 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:45.978075    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8fc7ae79-229c-4f3f-a83e-3b40b6d2ba96/init-config-reloader/0.log"
Oct 30 22:50:46.387385 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.387336    5561 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-hcxmd"
Oct 30 22:50:46.708704 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.708684    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 22:50:46.714527 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.714508    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 22:50:46.873792 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873701    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-data\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.873792 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873753    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istiod-ca-cert\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873796    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-socket\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873862    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-podinfo\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873907    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-credential-socket\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873948    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-token\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-token\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874013 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873999    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-credential-socket\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.873998    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-data" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.874280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874025    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-kube-api-access-p2np5\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-kube-api-access-p2np5\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874102    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-envoy\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874149    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-token\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-token\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874169    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-socket" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.874280 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874177    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-envoy\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874620 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874304    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-credential-socket" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.874620 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874344    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-credential-socket" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.874620 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874340    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istiod-ca-cert" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:46.874620 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874519    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-socket\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874620 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874580    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec4d573a-ec69-4842-b77f-8c3932e1efef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec4d573a-ec69-4842-b77f-8c3932e1efef-istiod-ca-cert\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874657    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-kube-api-access-d7vvx\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-kube-api-access-d7vvx\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874656    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-socket" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.874898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874715    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-certs\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.874898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874755    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-certs\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874785    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-podinfo\") pod \"ec4d573a-ec69-4842-b77f-8c3932e1efef\" (UID: \"ec4d573a-ec69-4842-b77f-8c3932e1efef\") "
Oct 30 22:50:46.874898 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.874842    5561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-data\") pod \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\" (UID: \"ce7446d0-d8a5-4e3f-9895-49698bfa488d\") "
Oct 30 22:50:46.875243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875005    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-certs" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.875243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875207    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-data" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.875243 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875221    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-certs" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.875429 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875298    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec4d573a-ec69-4842-b77f-8c3932e1efef-istiod-ca-cert" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:46.875493 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875465    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875493 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875488    5561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-certs\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875503    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875516    5561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-data\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875530    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875543    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875559    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875572    5561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-credential-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875585    5561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-workload-socket\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.875611 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.875599    5561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec4d573a-ec69-4842-b77f-8c3932e1efef-istiod-ca-cert\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.877234 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.877207    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-envoy" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.877539 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.877503    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-token" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.877641 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.877555    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-kube-api-access-d7vvx" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "kube-api-access-d7vvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.877641 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.877592    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-envoy" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.877727 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.877672    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-podinfo" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:46.877781 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.877765    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-kube-api-access-p2np5" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "kube-api-access-p2np5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.877819 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.877778    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-token" pod "ce7446d0-d8a5-4e3f-9895-49698bfa488d" (UID: "ce7446d0-d8a5-4e3f-9895-49698bfa488d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.878514 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.878500    5561 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-podinfo" pod "ec4d573a-ec69-4842-b77f-8c3932e1efef" (UID: "ec4d573a-ec69-4842-b77f-8c3932e1efef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:46.976424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976332    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.976424 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976379    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.976719 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976697    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2np5\" (UniqueName: \"kubernetes.io/projected/ec4d573a-ec69-4842-b77f-8c3932e1efef-kube-api-access-p2np5\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.976777 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976725    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.976777 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976740    5561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-token\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.976777 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976754    5561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce7446d0-d8a5-4e3f-9895-49698bfa488d-istio-envoy\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.976777 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976769    5561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7vvx\" (UniqueName: \"kubernetes.io/projected/ce7446d0-d8a5-4e3f-9895-49698bfa488d-kube-api-access-d7vvx\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.976902 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:46.976784    5561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec4d573a-ec69-4842-b77f-8c3932e1efef-istio-podinfo\") on node \"ip-10-0-135-39.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.404421 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.404389    5561 generic.go:356] "Generic (PLEG): container finished" podID="ec4d573a-ec69-4842-b77f-8c3932e1efef" containerID="dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923" exitCode=0
Oct 30 22:50:47.404853 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.404486    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" event={"ID":"ec4d573a-ec69-4842-b77f-8c3932e1efef","Type":"ContainerDied","Data":"dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923"}
Oct 30 22:50:47.404853 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.404504    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"
Oct 30 22:50:47.404853 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.404535    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn" event={"ID":"ec4d573a-ec69-4842-b77f-8c3932e1efef","Type":"ContainerDied","Data":"0b65835b9489d78c26086c7efd73d53932f9d331b623f21aa48b2ebfc2578b29"}
Oct 30 22:50:47.404853 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.404564    5561 scope.go:122] "RemoveContainer" containerID="dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923"
Oct 30 22:50:47.406302 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.406272    5561 generic.go:356] "Generic (PLEG): container finished" podID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" containerID="1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303" exitCode=0
Oct 30 22:50:47.406461 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.406446    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" event={"ID":"ce7446d0-d8a5-4e3f-9895-49698bfa488d","Type":"ContainerDied","Data":"1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303"}
Oct 30 22:50:47.406559 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.406543    5561 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88" event={"ID":"ce7446d0-d8a5-4e3f-9895-49698bfa488d","Type":"ContainerDied","Data":"e5390b0b1d52a3d77845bb085b9f359c4cdb327a17eb906eafbca15a2941ebe4"}
Oct 30 22:50:47.406654 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.406641    5561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"
Oct 30 22:50:47.420728 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.420706    5561 scope.go:122] "RemoveContainer" containerID="dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923"
Oct 30 22:50:47.421039 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:50:47.421014    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923\": container with ID starting with dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923 not found: ID does not exist" containerID="dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923"
Oct 30 22:50:47.421093 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.421052    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923"} err="failed to get container status \"dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923\": rpc error: code = NotFound desc = could not find container \"dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923\": container with ID starting with dc8e1b3f5c345c3ac16058cffd40b864760c12867c79b53c3a36e9823908f923 not found: ID does not exist"
Oct 30 22:50:47.421093 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.421077    5561 scope.go:122] "RemoveContainer" containerID="1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303"
Oct 30 22:50:47.429777 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.429755    5561 scope.go:122] "RemoveContainer" containerID="1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303"
Oct 30 22:50:47.430053 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:50:47.430034    5561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303\": container with ID starting with 1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303 not found: ID does not exist" containerID="1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303"
Oct 30 22:50:47.430105 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.430061    5561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303"} err="failed to get container status \"1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303\": rpc error: code = NotFound desc = could not find container \"1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303\": container with ID starting with 1b659219409c1ee2006098af6ed6b9ea05445228c5e7d5d0213749760a141303 not found: ID does not exist"
Oct 30 22:50:47.575487 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.575459    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"]
Oct 30 22:50:47.625993 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.625971    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88"]
Oct 30 22:50:47.691132 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.691065    5561 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"]
Oct 30 22:50:47.696733 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.696708    5561 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn"]
Oct 30 22:50:47.845829 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.845797    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce7446d0-d8a5-4e3f-9895-49698bfa488d" path="/var/lib/kubelet/pods/ce7446d0-d8a5-4e3f-9895-49698bfa488d/volumes"
Oct 30 22:50:47.846244 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:47.846231    5561 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec4d573a-ec69-4842-b77f-8c3932e1efef" path="/var/lib/kubelet/pods/ec4d573a-ec69-4842-b77f-8c3932e1efef/volumes"
Oct 30 22:50:48.109419 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:48.109331    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-66b4c7697-8m46d_f6990798-b558-45b4-86d2-ab0917893407/metrics-server/0.log"
Oct 30 22:50:50.205449 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:50.205419    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zqjjz_2be22202-fcae-4c90-b4ff-0eb07a3c41e0/node-exporter/0.log"
Oct 30 22:50:50.621731 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:50.621697    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zqjjz_2be22202-fcae-4c90-b4ff-0eb07a3c41e0/kube-rbac-proxy/0.log"
Oct 30 22:50:51.047617 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:51.047526    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zqjjz_2be22202-fcae-4c90-b4ff-0eb07a3c41e0/init-textfile/0.log"
Oct 30 22:50:52.980088 ip-10-0-135-39 kubenswrapper[5561]: E1030 22:50:52.980013    5561 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-q94hx" containerName="extractor"
Oct 30 22:50:55.189759 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:55.189704    5561 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-39.us-west-1.compute.internal" podUID="9a9ea5e58d330910c40e8b399ca003fd" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 22:50:56.889167 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:56.889130    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-h5pcb_c7eabbd5-c926-4eaf-9629-17b0da3ec706/prometheus-operator/0.log"
Oct 30 22:50:57.297996 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:57.297913    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-h5pcb_c7eabbd5-c926-4eaf-9629-17b0da3ec706/kube-rbac-proxy/0.log"
Oct 30 22:50:58.154760 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:58.154734    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cd5b84b84-5qqtf_d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/telemeter-client/0.log"
Oct 30 22:50:58.564775 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:58.564687    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cd5b84b84-5qqtf_d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/reload/0.log"
Oct 30 22:50:58.977488 ip-10-0-135-39 kubenswrapper[5561]: I1030 22:50:58.977453    5561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cd5b84b84-5qqtf_d919e55a-538a-48e6-9fe7-9e7d2bdd0ad7/kube-rbac-proxy/0.log"